This article was last updated on January 7, 2025, to include advanced techniques for securing Kubernetes ServiceAccounts, such as managing cross-namespace access, using short-lived tokens, and implementing RBAC for minimal permissions, along with simplified explanations to enhance clarity.

Introduction

What is a Kubernetes ServiceAccount?

In a simplistic way, an in-Kubernetes ServiceAccount is a special kind of entity an application is run as when sitting inside Pods to have their interaction securely with the Kubernetes API or whatever resource, which has permissions on what may or may not be accessed at whatever level.

TL;DR: In Kubernetes, ServiceAccounts are a sort of 'ID' card for your application that controls what your applications can and cannot do. And, after managing a couple of clusters of Kubernetes for these years, I learned that proper configuration of ServiceAccounts is essential in terms of security: it's actually giving the house key to the right person.

Steps covered in this article:

• What is a Kubernetes ServiceAccount?
• Service Account Creator Tool
• Understanding Service Accounts
• Understanding Default Service Accounts
• Key Use Cases of ServiceAccounts
• Granting Permissions to ServiceAccounts using RBAC
• Ensuring Secure Cross-Namespace Access
• Customizing ServiceAccounts in Pod Specifications
• Advanced Authentication with ServiceAccount Tokens
• Step-by-Step Guide: Creating a ServiceAccount
• Include a Section on FAQ
• Common Problems and Solutions

Service Account Creator Tool

Now, let's find the right configuration of ServiceAccount serving your use case. This interactive tool will guide you through an ideal setup depending on your security and application requirements:

What type of access does your application need?

Click to zoom

Understanding Service Accounts

For a moment, imagine that you operate a high-security building, and every visitor who comes in does not get unlimited access to each and every area. Well, ServiceAccounts act in the same way within your Kubernetes cluster-something like security badges that determine how far each different application can get in your cluster.

I remember when I first started with Kubernetes, I made the rookie mistake of using the default ServiceAccount for everything. Trust me, that is like giving everyone a master key to your building! Let me show you a better way.

ServiceAccounts are namespaced, so this is a good way to clean up access. You can think of namespaces like departments in your building: each has different security needs.

Understanding Default Service Accounts

Here is something that may get someone blind sided when first coming in: every namespace immediately gets a default ServiceAccount. As if every newly created department automatically has an access card with basic accesses.

But here's the catch - this default ServiceAccount is rather limited. It's the visitor badge that gets you into the lobby and that is all. Great for security, but your applications may well not have sufficient permissions to perform their tasks.

I learned this the hard way when one of our monitoring pods couldn't access the metrics it needed. The solution was to create a custom ServiceAccount with the right permissions.

Key Use Cases of ServiceAccounts

Let me share some real-world scenarios where ServiceAccounts shine:

• API Communications: Suppose your application needs to speak with the Kubernetes API. For that, a ServiceAccount can act as a trusted messenger in securing those communications. I do this heavily for our monitoring tools, which need to watch the state of pods.

• Cross-Namespace Access: Applications operating in one namespace sometimes need to peek into another. For example, our logging system in the 'monitoring' namespace needs to access pods in all other namespaces. A well-configured ServiceAccount does indeed make this possible and secure.

• External Service Authentication: Your pods may need to talk to an external service such as cloud APIs, which require some means of establishing trust. In my case, I use them for our backup system that needs to access cloud storage.

• Private Image Registries: Pulling images from any private registry? ServiceAccounts can hold the credentials securely. That saved us from hardcoding registry credentials in our manifests.

• CI/CD Integration: Want your CI/CD pipelines to deploy to Kubernetes? ServiceAccounts give them the secure access they need. We use this in our GitOps workflows.

Granting Permissions to ServiceAccounts using RBAC

Let's talk about RBAC: basically writing the security rulebook for your ServiceAccounts. I do it this way:

First, create a Role that defines what actions are allowed. For example:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: my-namespace
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "watch"]

Then, you bind this Role to your ServiceAccount:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: my-namespace
subjects:
- kind: ServiceAccount
  name: my-service-account
  namespace: my-namespace
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Ensuring Secure Cross-Namespace Access

One of the most challenging areas in securing Kubernetes is properly managing access across namespaces. This is one area where many teams struggle. I have seen them revert to using highly permissive configurations. There is a better way, and here it is:

Suppose you have a monitoring service in the 'monitoring' namespace that needs to watch pods in the 'production' namespace. Here's how to do it safely:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: production
  name: pod-viewer
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: monitoring-pod-viewer
  namespace: production
subjects:
- kind: ServiceAccount
  name: monitoring-service
  namespace: monitoring
roleRef:
  kind: Role
  name: pod-viewer
  apiGroup: rbac.authorization.k8s.io

Customizing ServiceAccounts in Pod Specifications

Time to get real-world - actually using your ServiceAccounts in pods. I'll always explicitly denote my ServiceAccounts within pod specs - even the usage of the default one. Makes what's going on regarding the security setup explicit to all readers:

apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  serviceAccountName: my-custom-sa
  containers:
  - name: main
    image: my-app:latest

Pro Tip: I learned to always document why a particular ServiceAccount is in use. Future you - or your teammates - will thank you!

Advanced Authentication with ServiceAccount Tokens

Let me share something that tripped me up initially: ServiceAccount tokens come in two flavors - short-lived and legacy. After a production incident where leaked legacy tokens caused us headaches, I now always use short-lived tokens:

apiVersion: v1
kind: Pod
metadata:
  name: token-test
spec:
  serviceAccountName: my-sa
  containers:
  - name: main
    image: my-app:latest
    volumeMounts:
    - name: token
      mountPath: /var/run/secrets/tokens
  volumes:
  - name: token
    projected:
      sources:
      - serviceAccountToken:
          path: token
          expirationSeconds: 3600

Step-by-Step Guide: Creating a ServiceAccount

Having used such environments in production setups, I will guide you through the process of creating and setting up a ServiceAccount.

Step 1: Create the Namespace (Optional)

If you're working in a new namespace:

kubectl create namespace my-app
kubectl config set-context --current --namespace=my-app

Step 2: Create the ServiceAccount

Create a file named service-account.yaml:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app-sa
  namespace: my-app
  labels:
    app: my-app
    environment: production

Apply it:

kubectl apply -f service-account.yaml

Step 3: Define Required Permissions

Create a file called role.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: my-app-role
  namespace: my-app
rules:
- apiGroups: [""]  # Core API group
  resources: ["pods", "services"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]  # Apps API group
  resources: ["deployments"]
  verbs: ["get", "list"]

Apply the role:

kubectl apply -f role.yaml

Step 4: Role Bind the ServiceAccount

Create a file, role-binding.yaml, containing:

Apply binding:

kubectl apply -f role-binding.yaml

Step 5: Test the Configuration

Check if the ServiceAccount was created:

kubectl get serviceaccount my-app-sa -o yaml

Checking the permissions:

kubectl auth can-i get pods --as=system:serviceaccount:my-app:my-app-sa

Step 6 - Using the ServiceAccount in Pod

Create a file named pod.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: my-app-pod
  namespace: my-app
spec:
  serviceAccountName: my-app-sa
  containers:
  - name: my-app
    image: my-app:latest
    volumeMounts:
    - name: sa-token
      mountPath: /var/run/secrets/kubernetes.io/serviceaccount
  volumes:
  - name: sa-token
    projected:
      sources:
      - serviceAccountToken:
          expirationSeconds: 3600
          audience: my-app

Apply the pod:

kubectl apply -f pod.yaml

Step 7: Security Best Practices

Security practices that I always follow include:

• Token Lifetime: Use short-lived tokens

serviceAccountToken:
     expirationSeconds: 3600  # 1 hour

• Namespace Isolation: Keep ServiceAccounts namespace-scoped unless absolutely necessary

# Avoid cluster-wide permissions when possible
   kubectl create clusterrole --help  # Think twice before using this

• Audit Regularly: Know who can access what.

kubectl auth can-i --list --as=system:serviceaccount:my-app:my-app-sa

• Minimal Permissions: Grant permissions minimal, but add as required.

rules:
   - apiGroups: [""]
     resources: ["pods"]
     verbs: ["get", "list"]  # Only what's needed
     resourceNames: ["app-*"]  # Restrict to specific resources

Include a Section on FAQ

What is the purpose of a Kubernetes ServiceAccount?

It gives a secure identity to pods while accessing Kubernetes resources and APIs.

How do you create a service account in Kubernetes?

You can use the command kubectl create serviceaccount <name> or define it in a YAML manifest.

What is the difference between a ServiceAccount and a User Account in Kubernetes?

ServiceAccounts are for applications running in pods, while User Accounts are for human users accessing the cluster.

How do you assign permissions to a ServiceAccount?

Use RBAC to bind roles to a ServiceAccount.

How to improve the security of ServiceAccount? Use short-lived tokens, implement least privileges, and isolate namespaces.

Common Problems and Solutions

• Pod Can't Reach API Server

# Check if token is mounted
   kubectl exec my-app-pod -- ls /var/run/secrets/kubernetes.io/serviceaccount
   
# Verify token permissions
   kubectl auth can-i --list --as=system:serviceaccount:my-app:my-app-sa

• Permission Denied Errors

# Check RoleBinding
   kubectl get rolebinding my-app-role-binding -o yaml
   
# Check actual permissions
   kubectl auth can-i get pods --as=system:serviceaccount:my-app:my-app-sa

• Token refresh not occurring

# Ensure projected volume is used
   volumes:
   - name: token
     projected:
       sources:
       - serviceAccountToken:
           expirationSeconds: 3600

Remember: it is always valid to check all your ServiceAccount settings in a test environment before going onto production. I learned the hard way.

Conclusion

After working with Kubernetes for years, I have learned that ServiceAccounts are the security guards of your cluster: strict enough to keep things secure, but flexible enough to let legitimate work happen. Start with the basics, understand the security implications, and gradually build up to more complex setups as your needs grow.

Remember: in security in Kubernetes, it's better to start off tight and loosen up if needed - rather than the other way around. Take this from experience: cleaning up after a security incident is way harder than preventing one!