DEVOPS 17 min

Kubernetes Security: PSA, RBAC & Zero Trust Networks

Take the leap into enterprise orchestration. Understand Kubernetes architecture: Nodes, Pods, Deployments, and Services. The foundation for deploying fault-tolerant applications at scale.

Written by Samir El Azem Crovetto • Senior Backend Engineer & DevSecOps

From Docker Compose to Kubernetes: Production Operational Realities

Docker and Docker Compose provide an exceptional developer ergonomics experience for local testing and single-node service deployments, as examined in our guide on production MySQL with Compose. However, when organizations scale toward distributed, mission-critical systems, Docker Compose exhibits structural limits: it lacks autonomous node failure remediation, lacks dynamic internal service mesh routing, and cannot coordinate zero-downtime rolling updates across heterogeneous hosts.

Kubernetes (K8s) overcomes these limitations by orchestrating multi-node server clusters into an unified compute fabric. Yet, this architectural power drastically magnifies the attack surface if deployed with factory-default configurations:

  • Default-Allow Flat Networks: By default in Kubernetes, any Pod in any namespace can initiate TCP/UDP connections to any other Pod across the cluster, permitting immediate lateral movement upon a single application breach.
  • *Overprivileged Containers (Privileged Pods):* Pods running as root (UID 0), with writable root filesystems, or retaining broad Linux capabilities (CAPSYSADMIN), paving the way for container escape vulnerabilities directly into the host operating system kernel, as analyzed in what is Docker.
  • Exposed ServiceAccount Tokens: Kubernetes automatically mounts a signed JWT representing the Pod's ServiceAccount identity at /var/run/secrets/kubernetes.io/serviceaccount/token. If an attacker breaches the application and the ServiceAccount holds broad API privileges, full cluster takeover becomes feasible.
  • Unrestricted API Server Ingress: Publicly exposed API servers lacking strict OIDC identity federation or running with permissive RBAC assignments granting cluster-admin to service automation.

Hardening Kubernetes requires a multi-tier defense-in-depth model spanning from API admission control down to eBPF kernel event tracing.

Defense-in-Depth Architecture in Kubernetes

Enterprise Kubernetes security relies on concentric layers isolating identity, network flow, and Linux kernel execution:

graph TD
    subgraph Control_Plane ["🛡️ Control Plane (API Server)"]
        User["Developer / CI/CD"]
        OIDC["OIDC / Cloud IAM Authentication"]
        RBAC["RBAC Authorization (Least Privilege)"]
        PSA["Pod Security Admission (Enforce: Restricted)"]
    end

    subgraph Data_Plane ["⚙️ Data Plane (Worker Node)"]
        subgraph CNI_Layer ["CNI Network Filtering (NetworkPolicy)"]
            NetPol["Default-Deny Ingress & Egress"]
        end

        subgraph Pod_Runtime ["Pod Isolation (Hardened Runtime)"]
            SecComp["Seccomp Profile: RuntimeDefault"]
            NonRoot["runAsNonRoot: true (UID >= 10000)"]
            ReadOnly["readOnlyRootFilesystem: true"]
            DropCaps["capabilities.drop: ['ALL']"]
        end

        subgraph Kernel_Audit ["Runtime Threat Detection (eBPF)"]
            Falco["Falco eBPF: Shell Spawn & Anomaly Engine"]
        end
    end

    User -->|TLS 1.3 + Bearer Token| OIDC
    OIDC --> RBAC
    RBAC --> PSA
    PSA -->|Validated Manifest| CNI_Layer
    CNI_Layer -->|Whitelisted Traffic| Pod_Runtime
    Pod_Runtime --> Kernel_Audit
    Kernel_Audit -.->|Security Incident Alert| SIEM["SIEM / SOC Alerting"]

    classDef control fill:#e1f5fe,stroke:#0288d1,stroke-width:2px;
    classDef data fill:#e8f5e9,stroke:#388e3c,stroke-width:2px;
    classDef runtime fill:#fff3e0,stroke:#f57c00,stroke-width:2px;
    classDef audit fill:#ffebee,stroke:#d32f2f,stroke-width:2px;

    class User,OIDC,RBAC,PSA control;
    class CNI_Layer data;
    class SecComp,NonRoot,ReadOnly,DropCaps runtime;
    class Falco,SIEM audit;

1. Pod Security Standards (PSS) & Pod Security Admission (PSA)

Following the removal of legacy PodSecurityPolicies (PSP) in Kubernetes 1.25, the built-in standard is Pod Security Admission (PSA), which categorizes workloads into three governance profiles:

  1. Privileged: Unrestricted. Intended strictly for low-level node daemons (CNI network drivers, CSI storage agents).
  2. Baseline: Enforces basic security guardrails with minimal configuration adjustments.
  3. Restricted: Implements the most stringent hardening controls aligned with CIS Kubernetes Benchmarks.

Namespace-Level PSA Enforcement

To ensure that insecure containers cannot be deployed into production namespaces, annotate namespaces with the enforcement labels:

apiVersion: v1
kind: Namespace
metadata:
  name: production-apps
  labels:
    # Reject deployment of any Pod violating the Restricted standard
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest
    # Warn operators during kubectl apply if templates violate guidelines
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/warn-version: latest
    # Log audit events for compliance analysis
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/audit-version: latest

Hardened Production Pod Manifest

Under the restricted profile, every deployed Pod must define its securityContext parameters explicitly:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: secure-backend-service
  namespace: production-apps
spec:
  replicas: 3
  selector:
    matchLabels:
      app: backend-service
  template:
    metadata:
      labels:
        app: backend-service
    spec:
      # Disable ServiceAccount token auto-mounting if the app does not call the K8s API
      automountServiceAccountToken: false
      
      securityContext:
        runAsNonRoot: true
        runAsUser: 10001
        runAsGroup: 10001
        fsGroup: 10001
        seccompProfile:
          type: RuntimeDefault

      containers:
        - name: application
          image: ghcr.io/enterprise/backend-service:v1.4.2
          imagePullPolicy: IfNotPresent
          
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop:
                - ALL

          resources:
            limits:
              cpu: "1"
              memory: "1Gi"
            requests:
              cpu: "250m"
              memory: "512Mi"

          volumeMounts:
            # Ephemeral in-memory scratch space for required temporary file operations
            - name: tmp-volume
              mountPath: /tmp

      volumes:
        - name: tmp-volume
          emptyDir:
            medium: Memory
            sizeLimit: 64Mi

Key security parameters:

  • allowPrivilegeEscalation: false: Blocks binaries with the setuid bit from elevating execution privileges within the container.
  • readOnlyRootFilesystem: true: Locks down the root filesystem as read-only. Even if an attacker finds an application exploit, they cannot download exploit toolkits (curl, nmap), compile shellcode, or alter system binaries.
  • capabilities.drop: ["ALL"]: Strips all administrative Linux kernel capabilities (such as CAPNETRAW or CAPSYSPTRACE).
  • seccompProfile: RuntimeDefault: Enforces native Linux system call filtering to restrict dangerous or unused kernel calls.

2. Network Microsegmentation with NetworkPolicies

By default, Kubernetes network topology is completely open: any Pod can communicate with any other Pod. To establish a Zero Trust baseline, the foundational step in every namespace is applying a Default-Deny policy across both Ingress and Egress.

Policy 1: Global Default-Deny Policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: production-apps
spec:
  podSelector: {} # Selects all pods in the namespace
  policyTypes:
    - Ingress
    - Egress

Once applied, all Pods in the namespace are isolated from both inbound connections and outbound internet traffic, neutralizing lateral reconnaissance.

Policy 2: Granular Service Whitelisting

Next, explicitly authorize required traffic channels:

  • Ingress: Inbound TCP on port 8080 originating strictly from pods labeled role: ingress-controller located in the ingress-nginx namespace.
  • Egress: Outbound DNS queries to Kubernetes CoreDNS (kube-dns on port 53 UDP/TCP) and database traffic to the private PostgreSQL cluster on port 5432:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-service-policy
  namespace: production-apps
spec:
  podSelector:
    matchLabels:
      app: backend-service
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: ingress-nginx
          podSelector:
            matchLabels:
              app.kubernetes.io/name: ingress-nginx
      ports:
        - protocol: TCP
          port: 8080
  egress:
    # Allow DNS resolution
    - to:
        - namespaceSelector: {}
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53
    # Allow connection to private database tier
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: database-tier
          podSelector:
            matchLabels:
              app: postgres-cluster
      ports:
        - protocol: TCP
          port: 5432

Critical Implementation Note: NetworkPolicies are only enforced if the cluster utilizes a CNI plugin with policy filtering support, such as Cilium, Calico, or AWS VPC CNI with network policy engine enabled. Default basic bridge plugins do not enforce filtering.

3. RBAC and Workload Identity: Eliminating Static Secrets

Role-Based Access Control (RBAC) should favor namespace-scoped Role bindings over cluster-wide ClusterRole permissions, strictly enumerating verbs and resources:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: production-apps
  name: deployment-operator
rules:
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]

Workload Identity (AWS EKS Pod Identity / IRSA)

In cloud environments, containerized microservices frequently interact with cloud services like Amazon S3, DynamoDB, or Secrets Manager.

An antipattern is creating long-lived static AWS credentials (AWSACCESSKEYID and AWSSECRETACCESSKEY) and storing them in Kubernetes Secrets. If the cluster is compromised, the keys leak and persist indefinitely.

The enterprise standard is Workload Identity (EKS Pod Identity or IAM Roles for Service Accounts - IRSA):

  1. A Kubernetes ServiceAccount is bound to an AWS IAM Role via an OIDC token minted by the cluster API server.
  2. The AWS SDK within the container automatically assumes the IAM role via sts:AssumeRoleWithWebIdentity.
  3. Credentials are short-lived, rotated automatically every hour, and never written to disk.

4. Runtime Threat Detection with Falco eBPF

Even with static admission controls and network microsegmentation, runtime behavior must be monitored continuously. Falco is the CNCF standard for runtime security detection in Kubernetes, leveraging Linux eBPF (Extended Berkeley Packet Filter) probes inside the operating system kernel.

Falco intercepts system calls in real-time and evaluates rules for abnormal behavior:

  • Interactive shell execution inside production containers (exec of /bin/bash or /bin/sh).
  • File writes to system binary paths (/bin, /sbin, /usr/bin).
  • Unexpected outbound network socket connections to external IP ranges.
  • Anomalous ServiceAccount token reads.

Example Falco rule to alert on container shell execution:

- rule: Terminal shell in container
  desc: A shell was spawned inside a container with an attached terminal
  condition: >
    spawned_process and container
    and shell_procs and proc.tty != 0
    and container.name != "debug-sidecar"
  output: >
    Critical Alert: Interactive shell spawned in container
    (user=%user.name pod=%k8s.pod.name ns=%k8s.ns.name
     cmd=%proc.cmdline image=%container.image.repository)
  priority: WARNING
  tags: [container, shell, mitre_execution]

Events are streamed in real time to corporate SIEM platforms or automated incident response pipelines, enabling security teams to isolate compromised Pods immediately.

Enterprise Kubernetes Hardening Checklist

  • [x] Pod Security Admission configured with enforce: restricted across all production namespaces.
  • [x] Containers hardened with readOnlyRootFilesystem: true, runAsNonRoot: true, and capabilities.drop: ["ALL"].
  • [x] Automatic ServiceAccount token mounting disabled (automountServiceAccountToken: false) where API access is unnecessary.
  • [x] Network microsegmentation implemented with default-deny-all NetworkPolicies.
  • [x] Static cloud credentials replaced by federated Workload Identity / EKS Pod Identity.
  • [x] State-aware CNI plugin (Cilium or Calico) deployed and actively enforcing policies.
  • [x] Continuous runtime auditing configured via Falco eBPF for immediate breach detection.

If you need to audit your organization's cloud architecture, implement Zero Trust bastions, or harden container clusters and Kubernetes workloads, explore my Cybersecurity Consulting Services or reach out directly via About Me.