Container orchestration is moving from hype to mainstream quickly, with Kubernetes leading the charge. We have seen a lot of progress and exciting additions this year. But as the field matures, the requirements on the surrounding ecosystem also raise. It is not enough to “just” do container orchestration. You also need a logging solution, metrics collection, ingress support etc… Now the question arises: should you do this yourself or utilize an existing solution, like OpenShift?
There are two fundamental solutions to this: either you take the time and build everything exactly the way you want it, or you go with a Kubernetes distribution that already includes what you need. To help you choose, think about these questions:
- Do you need tight control over all the deployment details?
- Do you have any requirements on the environment surrounding Kubernetes?
- Where does your business value come from?
- Does it come from the specialized custom environment?
- Does it come from the applications that you are running in Kubernetes?
In this post we are going to take a closer look at how the OpenShift family of Kubernetes distributions differ from Kubernetes vanilla. Quite a lot has happened in the last year, and many old articles are just not accurate any more. So let’s find our way through this jungle, shall we?
What is OpenShift?
First of all, what members does the OpenShift family have and what are they actually called this year? Here is the list:
- OKD (Origin Kubernetes Distribution), previously known as OpenShift Origin, is the open source project behind Red Hat OpenShift.
- OpenShift Online is a public cloud service hosted by Red Hat.
- OpenShift.io is a SaaS solution on top of OpenShift Online which basically gives you the complete environment you need for writing code and deploying software, right in your browser.
- OpenShift Dedicated is a private cloud service hosted by Red Hat.
- OpenShift Container Platform is Red Hat’s PaaS for on-premise or cloud deployments. It was previously known as OpenShift Enterprise.
- OpenShift Container Engine is the newcomer of the family and really just a trimmed down version the Container Platform.
With that out of the way, what makes OpenShift different from vanilla Kubernetes? This is a bit tricky to answer since the two are evolving quite rapidly. OpenShift recently included the Ingress resource, for example, which previously was a difference. In short, the Kubernetes core in OpenShift is not that far from vanilla. The real difference lies in the additional services, configuration and administration tools.
OpenShift includes several service templates.The Installer
It all starts with the installer, a collection of Ansible playbooks and roles. But it’s so much more than that! There are playbooks and roles for everything from scaling up the cluster to upgrading it to the next release. The same tool you use for making the first deployment can later be used to add the EFK stack (Elasticsearch, Fluentd and Kibana) or set up GlusterFS for persistent volumes. All that by changing a few variables and then executing a playbook.
The installer may give you super powers for managing your deployment, but it also comes with some restrictions. For example, only RPM distributions like Red Hat Enterprise Linux and CentOS are supported. Additionally, you are limited to the configuration options that the installer exposes and supports. This means, for example, that you cannot use an external etcd cluster (from v3.10).
Additions to Kubernetes
OpenShift includes some additions to Kubernetes. Here are some highlights:
- Built in container registry
- Metrics and monitoring with Hawkular or Prometheus
- Log aggregation with Elasticsearch and Kibana
- Multi tenancy
- CI/CD and devops work flows
Your can of course add most, if not all of these “extras” to any Kubernetes cluster given enough time and work. Besides, OpenShift can very well be deployed without some of these additions if you so wish. But that is beside the point. If you can avoid inventing the wheel again, do it! And of course: don’t waste money or resources on running or configuring things you don’t need.
Infrastructure and Resources
Infrastructure nodes is something you are bound to run into sooner or later if you wish to manage your own cluster. The idea is simple, reserve some nodes for “infrastructure loads” such as log aggregation and monitoring. Done properly, this should allow you access to important data (such as logs) even in case of worker node failures. However, configuring this yourself will inevitably require some time and effort which could potentially be better invested elsewhere. Not necessarily, but potentially.
Another thing that OpenShift takes care of for you is resource requests and capacity requirements. There are of course some requirements specified for running Kubernetes also, but they are only for a minimal cluster and are not really checked either. The OpenShift installer on the other hand will check that all nodes have enough disk space and memory before proceeding. Additionally, all the pods added by the installer have resource requests and limits which should give a more stable experience. Without limits a pod could potentially hog up all the memory on a node and bring it down completely.
Multi Tenancy and Auth
Finally, OpenShift gives you the web console along with logging and metrics for all components of the cluster with a single sign on for everything! You can use your existing LDAP system for authentication and users are properly separated and cannot access each others projects. It is also easy to set up resource quotas for your users to make sure that they stay within bounds.
The login page for OpenShift Web Console.The Future?
Red Hat’s acquisition of CoreOS will eventually lead to a merge of features from the Tectonic stack into OpenShift. The vision is that upgrade pains should be a thing of the past in version 4.0. Red Hat has hinted at “full stack automation” coming early next year.
Another trend is that OpenShift is moving closer to Kubernetes and adopting successful tools from the Kubernetes ecosystem. For example, Prometheus will replace quite a few custom components in OpenShift, such as Hawkular and the CloudForms Management Engine.
Conclusion
Does this mean that you should jump on OpenShift immediately and just forget about Kubernetes? No. There are use cases for both and you must consider what you actually need. Some people like to buy their furniture from IKEA with “some assembly required”, others prefer no assembly required. And then we have the extremes who enjoy building the whole house from scratch.
No matter which way way you go, you should be aware of the alternatives before you start making decisions. We at Elastisys are familiar with both Kubernetes “the hard way” and OpenShift, as well as several other Kubernetes distributions. Don’t hesitate to reach out if you would like some advice on which way to go.
P.S. Remember to follow us on LinkedIn for more posts like this!
