A while ago we looked at Kubernetes vanilla and OpenShift to try to figure out when to use which solution. In this post we will take a closer look at the alternatives available and why you may want to go through the hassle of setting things up in Kubernetes rather than relying on built in OpenShift functionality.
There is a reason why OpenShift includes a lot of functionality on top of what Kubernetes provides out of the box. It is simply needed in many cases. So why would you choose Kubernetes then? Why invest time and money on setting up tools needed to run your cluster? Read on to find out!
Feature Comparison
Let’s start by taking a look at the Kubernetes alternatives for built in OpenShift functionality. The table below contains some features that appear in one way or another in OpenShift. In the Kubernetes column, you can see what would be needed to get similar functionality in Kubernetes.
| Feature | Kubernetes | OpenShift |
|---|---|---|
| Image Registry | Docker | Built in: OCR |
| Monitoring | Prometheus | Built in: Hawkular |
| Log aggregation | EFK stack | Built in: EFK |
| Multi tenancy | RBAC, network policies and container permissions | Built in |
| Infra nodes | Taints and tolerations | Built in |
| Package management | Helm | Templates |
| Routing | Ingress and Ingress Controller | Built in: Routes |
| Certificate management | Cert-manager | Built in/manually managed |
| CI/CD | Jenkins | Built in |
| Auth | dex | Built in |
Most of these features have direct counterparts in the Kubernetes column, but will of course require a bit more configuration and attention. Out of these examples, multi tenancy is probably the hardest (and most risky) thing to build yourself. So if you do need it, make sure you know what you are doing before ditching OpenShift. If, on the other hand, you work in a single tenant environment, it may be confusing and frustrating to have to deal with the “features” that you don’t need. Let’s take a concrete example!
Note: as pointed out by u/varesa on r/openshift and u/kubidu on r/kubernetes over at reddit, OpenShift has promised to use Prometheus in the upcoming version 4.0 instead of the current Hawkular/Heapster/Cassandra stack. Here is a list of major changes coming in 4.0. Thanks for the feedback!
Example: Small Enterprise with Legacy Services
Some assumptions:
- A single Ops team is the only tenant.
- Some legacy (non containerized) services are running outside of the cluster.
- A monitoring and logging solution is already in place (for the legacy services), with dedicated hardware.
- You plan on utilizing pre-existing Helm charts for several applications in the cluster.
In this specific scenario, OpenShift may not make sense. You would probably want to utilize the monitoring solution that you already have. And multi tenancy may get in your way when you try to manage the whole cluster as one unit. Additionally, Helm is not compatible with OpenShift as of now, without modifications to the cluster. This is because of the server component (Tiller) that requires quite a lot of permissions in order to manage your deployments. Note, however, that this will not be the case once Helm v3 is out.
Certificates and Routing
Another consideration is certificate management and routing. OpenShift is great for the more traditional way of dealing with certificates. You buy a certificate that is valid for a long time and that covers the whole domain. With this root certificate it is then easy to request routing for any sub domain and have new certificates signed by the root. Be mindful about the multi tenant nature of this though: OpenShift will give ownership of the sub domain to the first project (namespace) that requests a route/ingress for it.
Things can be a bit more flexible in Kubernetes. You would have to deploy an ingress controller and the cert-manager to get this functionality, but that is simple given that you use Helm. Check out the charts for nginx ingress controller and cert-manager. With these components added to the cluster, you can set up Let’s Encrypt as a certificate issuer. This will allow you to create Certificate resources in your cluster, or even use ingress-shim to automatically generate certificates for your Ingress resources. Additionally, cert-manager will also take care of renewing your certificates when needed.
Continuous Integration and Delivery
We have one more thing to consider: CI/CD. OpenShift comes with S2I built in along with the powerful concept of Image Streams. Will this force you to abandon the reasoning above and give in to OpenShift? Fear not, it is possible to set up a similar flow in Kubernetes using Jenkins. This guide will show you how to set up a flow where a new deployment is rolled out automatically whenever a new commit is pushed. It will not give you Image Streams, but if you set up your own Docker registry you can still get quite good control over what images you are using. And if automated updates is what you need, check out Keel and try it out in your cluster with this chart.
Other Considerations
Everything is not about the features you get with one system or the other. Sometimes it makes sense to follow the stream in order to enjoy the community support. Or on the other hand, you may want the security that comes with enterprise support.
OpenShift obviously comes with the backing of Red Hat, both as a hosted service or with on premise support. There are also several hosted Kubernetes services, for example GKE by Google. As for the community of the two projects, I think it is safe to say that Kubernetes is the clear winner. (See figure below.)
This means that it is easier to find people familiar with Kubernetes than OpenShift. A large community is usually good at providing free support as well. However, the OpenShift community also includes professionals paid by Red Hat. And you do not necessarily have to pay for enterprise support in order to get some of their help. If solving your issue is good for the paid product they will definitely be interested.
Conclusion
OpenShift includes a curated collection of services and tools on top of a customized Kubernetes cluster. By its very definition, this curated collection is not something you have much control over. You cannot easily switch one of the chosen components for another similar component. So some of the tools that you get with OpenShift may be dated or just a bit different from the current de-facto standard in the Kubernetes world. On the other hand, the security provided by enterprise support can be compelling. Besides, sometimes you do not want to spend too much time deciding between options.
There are cases when it makes sense to go with Kubernetes rather than OpenShift. Just make sure that you are using the right tool for the job! We at Elastisys are happy to help you which ever way you choose to go. As one of very few certified Kubernetes service partners as well as Red Hat Advanced Business Partner, you cannot go wrong with advice from us!
Remember to follow us on LinkedIn for more content like this.
