Elastisys provides fully-managed Kubernetes platforms with a rather unique responsibility model: we take responsibility for not just the Kubernetes control plane, but for the entire platform. This implies two things: customers put a lot of faith in our ability to securely operate their platforms, and that we have very high standards to earn that faith.
In this article, we explain one very important aspect of how we earn that faith: how we work with access credentials on all levels: cloud infrastructure provider credentials, Kubernetes API access, and SSH keys for system administration. By being this transparent, even to the public, we show why our customers are willing to lean on us and the service we provide.
Different types of access credentials
Elastisys works with multiple cloud infrastructure providers to offer our managed Kubernetes platform services. Since we do not have our own cloud infrastructure, we have developed tools and processes that let us conveniently work with all providers. In this section, we will describe the three types of credentials that we use during operations:
- Cloud infrastructure provider credentials, for setting up the cloud infrastructure both when we create a Kubernetes cluster and when the Kubernetes cluster itself needs to provision more infrastructure, such as virtual hard drives for use as block storage.
- Kubernetes API access, for when our platform administrators need to log in and carry out operational tasks against the Kubernetes clusters we manage, such as upgrading the Kubernetes platform to keep it running smoothly and secure against attacks.
- SSH keys, for when our administrators need to log in to the underlying virtual machines, i.e., the Kubernetes worker or control plane nodes, to fix tricky issues such as networking malfunctions.
Cloud infrastructure provider credentials
For each cloud infrastructure provider, each individual Elastisys platform administrator has their own individual account. For some, this has to be done on a per-project level, meaning that for each customer Kubernetes platform, we have our own individual accounts.
For infrastructure providers where multi-factor authentication is possible, we make use of that. For those where it isn’t, we have strong passwords that adhere to the password recommendations of the Swedish Civil Contingencies Agency (MSB) (link in Swedish).
The Kubernetes platforms need access to the underlying cloud infrastructure for automation purposes: to request block storage, to start and stop servers, and such. For this, the credentials it gets to use are at rest, stored encrypted in git repositories in such a way that only our specifically authorized personnel (the platform administrators) can read them, using SOPS encryption with personal GPG keys. For live automation use, the credentials are stored in Kubernetes Secrets, because this is what the Kubernetes cloud controller managers depend on. These are protected from even our customers via strong Kubernetes RBAC protection. We go to the utmost level to protect these, which is one of the reasons why we are strict about maintaining a clear divide between what access our customers can have. We also take proactive measures to ensure that no information can be disclosed via e.g. misconfigured privileged components in the platform (such as mitigating the NGINX Ingress Controller bugs that scared the community a while ago).
Kubernetes API access
Elastisys, as a company, uses Google Workspace as our company-wide Identity Provider (IdP).. We have configured it such that it requires strong passwords, and that we use multi-factor authentication. In all our platforms that we manage for customers, the Kubernetes control plane is configured such that it uses Dex IdP, the federated identity provider. For this, we use OpenID Connect to interface with Google Workspace. Of course, only members of our organization that should have platform administration rights are in the right group in our Google Workspace. Thus, we have Google’s secure handling of logins to rely on for administrative access to the Kubernetes API.
For customer logins, Dex trusts only the customer’s IdP.
SSH keys
Elastisys platform administrators only use SSH keys for logging in to virtual machines, never passwords. Password logins are disabled in the SSH daemon.
The SSH keys are tied to our individual administrators: if one leaves the company, they will not be able to log in any longer. Each key follows our key length policy in terms of algorithm choice, length, and thus strength.
The public part of the asymmetric SSH keys are distributed “everywhere”, to enable them to be used for logging in to each virtual machine. The private part is stored on Yubikeys, the personal Hardware Security Modules (HSM). This means that the private cryptographic part is never accessible even to our own administrators, but instead, they have to physically have access to both the HSM itself and knowledge of the PIN that unlocks and permits access to it.
Summary
By adhering to the best practices of the field regarding how to work with credentials on all levels, Elastisys has a proven track record by which it earns the trust to manage Kubernetes platforms even for society’s most critical entities, as defined by NIS2. This article has described our current practices. They are of course subject to change, because as the threat landscape changes, so do our practices.
If you want us to manage your Kubernetes platform, get in touch. And if you want to work with a company whose mission is to deliver high-end, premium products and services to the market in the areas of open source, cloud native operations, security, and compliance… we’re hiring.
