Kubernetes Security Best Practices – Security Boulevard

Kubernetes is an open source orchestration platform for containerized workflows. It is the best way to manage – or orchestrate – large clusters of containers at scale. Kubernetes, sometimes abbreviated as K8s, helps you efficiently manage clusters of hosts running Linux containers. 

In the age of containers, Kubernetes has become a popular open source project and key building block for modern tech infrastructure. Containers need to be managed, and Kubernetes helps with tasks such as scheduling, load balancing, and distribution, while making sure container workloads run as intended. With Kubernetes, you can deploy and manage cloud-native applications anywhere independent of cloud-specific requirements, avoiding infrastructure lock-in.

Kubernetes, which is Greek for helmsman or pilot, is exploding in popularity. According to a survey of the Cloud Native Computing Foundation (CNCF) end user community, Kubernetes use has increased from 58% in 2018 to 91% in 2020. Over the past four years, Kubernetes job searches have increased 2,125% according to Tech Republic, and the CNCF has recently introduced a Kubernetes security certification.

Kubernetes and containers are everywhere. With their rise in popularity, security should be a primary concern and not an afterthought.

Though Kubernetes has several built-in security advantages – for example, container images are usually replaced, not patched or updated which leads to better version control – its increase in use means more opportunity for exposure. Because Kubernetes is a relatively new and complex technology without enough talent to fully manage and secure it, it is also highly attractive to hackers.

When addressing Kubernetes security, it is nearly impossible to separate the Kubernetes cluster from the other layers of cloud computing. These layers include the application or code itself, the container image, the Kubernetes cluster, and the cloud or networking layer as shown in the image below. Each layer builds on the previous layer, and all layers must be secured to achieve defense in depth.

Cloud Native Security Layers

Another way to think more specifically about Kubernetes security from a developer point of view is to look at it across three distinct stages of the DevOps pipeline:

• Creating the container image during the build

• Configuring the Kubernetes infrastructure during deployment

• Securing network communication in production

Kubernetes Security During Build

Kubernetes security begins in the build stage when writing the code that becomes the container image. Although Kubernetes isn’t necessarily involved during the build and packaging stage, a secure environment starts with assessing the risk associated with any code being deployed. Application security best practices and testing are important here, and any effort to shift security left will pay dividends by avoiding future problems in deployment and production.

In this stage, you need to make sure that your container image is up to date, free from security vulnerabilities, and is compliant with your licensing policies. Containers are built-in layers, and often have no canonical list of what open source components have been included in the final build of the container image. To address this, you need to scan containers to determine which open source libraries are in play inside the application, including going through any dependencies that are brought in through a package manager like Maven or npm. By discovering and identifying all open source components in play, you can identify any known vulnerabilities, establish which images are at risk as new vulnerabilities are discovered, and track the licenses associated with specific libraries. Tracking license compliance may not be a security risk, but it can have a downstream effect like exposing proprietary code if an incompatible license is included.

Ensuring your base images are regularly scanned is the first step to securing your Kubernetes environment. 

WhiteSource Report – DevSecOps Insights 2020 Download Free
Report

Kubernetes Security During Deployment 

When deploying Kubernetes, you are given a powerful set of controls to secure clusters and their applications. Configuring these controls is not always easy. It requires an intimate knowledge of Kubernetes as well as the deployment’s security requirements. Don’t use defaults! It only leads to unnecessary (and perhaps even negligent) exposure. At the very least, you should limit the scope of user permissions, limit direct access to Kubernetes nodes, and create network segmentation so that containers communicate only with the containers they are supposed to.

Scanning at this stage remains of paramount importance. Container images must be built using secure base images. You need to implement continuous security vulnerability scanning and regularly apply security updates to your environment. Scanning is not a one-off event. In addition, when launching containers in your Kubernetes environment, only use images from image registries on allow lists.

Kubernetes Security in Production

Application security in runtime is very different from the build stage. Notably, network security is more complex. Best practices for network security in Kubernetes go beyond basic networking and leverage the container network interface (CNI) to implement a more robust networking layer that includes either multi-tenant support, network policies, or both. 

With a multi-tenant network, each namespace within the Kubernetes cluster gets a private addressable subnet and can only access pods in other namespaces that are specifically exposed as services. Open vSwitch is a great example of a networking layer for Kubernetes that has multi-tenant support.

Network policies in most next-generation networking layers are available to be deployed via Kubernetes. Project Calico is an example of a widely used networking layer that has been built from the ground up to support network policies with containers. Network policies allow cluster administrators to predefine network access control lists that control which ports are accessible and which services can be accessed. 

While this is far from an exhaustive list, here are some best practices for Kubernetes security at various stages to get you started.

Kubernetes Security During Build

• Scan your image and source code – As with any application, implementing application security testing best practices of using various scanning tools such as SAST, DAST, IAST, or SCA will help ensure your code is as secure as possible.

• Don’t forget about open source – Open source code is in almost every proprietary software offering on the market, making up approximately 60%-80% of all software codebases. If you’re not actively managing your open source code, you’re not secure.

• Implement workflows to remediate vulnerabilities – Automate your workflows to ensure that vulnerabilities are remediated as soon as possible and code is secure.

• Use updated images – This one goes without saying, but make sure you’re using the most updated image available as it will likely be the most secure image available. Destroy any outdated images; it’s not really possible to patch them.

Kubernetes Security During Deployment

• Don’t deploy images from unknown sources – If you don’t know the provenance of an image, you can’t trust it.

• Scan images during deployment – You might find new vulnerabilities that have been disclosed since your image was last scanned in development. 

• Use image admission controls – Admission controls stop an image if it violates your org’s security policies and prevent a container from running with greater privileges than it needs for its tasks. 

• Limit privileges used by a container – Giving a container broad permissions can be problematic if a malicious user gets control of it. Limit your exposure by limiting a container’s privileges and never run a container’s privilege flag.

• Don’t use defaults – Enable role-based access control (RBAC) in your container orchestration to regulate access based on the roles of individual users within your organization.

Kubernetes Security in Production

• Scan images in production – Again, we can’t stress this enough: new vulnerabilities are disclosed all the time. Make sure you’re meticulous about continuously scanning your images.

• Secure the network with a container firewall – You can secure and inspect your network using a container firewall, which applies traditional network security techniques to the cloud-native Kubernetes environment.

• Implement network security policies – Keep an eye on your network traffic flows by creating and defining cluster network policies that allow you to control network access in and out of your containerized applications. 

• Establish security boundaries using namespaces – Namespaces are virtual clusters in Kubernetes. They allow you to divide cluster resources between multiple users.

• Create a security policy for pods – Pods, the smallest execution unit in Kubernetes, are used to organize containers. Pod security policies can be used to address issues such as a container running as root. 

• Analyze pods in the same deployment – When replicated, pods should exhibit identical behavior. Monitor and analyze pods. If you see them behaving differently, you need to investigate immediately to ensure no breach has occurred.

Securing Kubernetes

• Update Kubernetes frequently – Kubernetes support covers only the most three recent versions. If you’re not updating Kubernetes regularly, you will miss out on patches for newly disclosed security vulnerabilities. As a bonus incentive, new security features are added in every quarterly Kubernetes update. Don’t expose your organization because of a basic oversight.

• Kubernetes API – Do not leave the Kubernetes API publicly exposed, and only allow access via an internal network or VPN.

Kubernetes security context defines privileges and access control settings for pods and containers. The security context establishes the security parameters that are assigned to each pod or container. Think of it as a set of rules for determining how and by whom a given resource should be accessed. And remember: don’t run as root!

The following table shows some of the most important Kubernetes security context parameters.

Kubernetes Security Context Settings

One of the challenges with Kubernetes and containers is a lack of visibility, which makes it harder to remediate vulnerabilities. Because images are continuously spun up and taken down, scans are more difficult to complete as the system is in constant flux. 

When looking for Kubernetes and containers security tools, look for tools that were built to handle the range of threats specific to Kubernetes. Prioritize tools and vendors that offer native support with your chosen Kubernetes distribution. Also look for tools that scan pods deployed to production for vulnerabilities or compliance issues and that enforce policies automatically.

Organizations are continuously updating and upgrading the Kubernetes environment as the technology matures. Periodic audits to determine the health and security of your Kubernetes deployment are recommended. When completing an audit, focus on the following areas:

A comprehensive Kubernetes security audit helps you prevent known exploits.

In their complexity, cloud-native environments like Kubernetes present new and evolving challenges when implementing security best practices and meeting internal security policy and compliance goals. Using the tips laid out here as a starting point, you can begin to harden your Kubernetes environment and reduce attack surfaces. Through effective security best practices, you can gain both visibility and control over every layer of your Kubernetes deployment.

*** This is a Security Bloggers Network syndicated blog from Blog – WhiteSource authored by Julie Peterson. Read the original post at: https://resources.whitesourcesoftware.com/blog-whitesource/kubernetes-security