Kubernetes Multi-Tenancy: The Hard Parts and How to Actually Do It

Kubernetes Multi-Tenancy: The Hard Parts and How to Actually Do It

“We run multiple teams on one Kubernetes cluster” is something I hear a lot. When I ask how they handle isolation between teams, the answer is usually “namespaces and RBAC.” This is better than nothing, but it’s not multi-tenancy in any meaningful security sense.

Real multi-tenancy — where teams can’t affect each other’s workloads, can’t see each other’s resources, and can’t escape their resource quotas — requires significantly more than namespaces. Here’s what the full picture looks like.

The Problem with “Just Namespaces”

Namespaces provide namespace-scoped resource isolation:

  • A pod in namespace A can’t directly access a pod in namespace B via Kubernetes API
  • RBAC can restrict who can do what in each namespace
  • NetworkPolicy can restrict network traffic between namespaces

But namespaces don’t provide:

  • Node isolation: Pods from different tenants run on the same nodes
  • Kernel isolation: Processes share the kernel; kernel exploits can escape namespace boundaries
  • Resource limit enforcement: Without ResourceQuota, one tenant can consume all cluster resources
  • API server access control: Without care, tenants can discover each other’s resources
  • Secret visibility: Cluster-scoped resources (ClusterRole, StorageClass) are visible to all
  • Control plane isolation: A particularly bad workload can impact control plane stability

The Isolation Spectrum

Before building, decide which tier of isolation you need:

Tier Isolation Level Mechanism Use Case
Namespace Soft Namespaces + RBAC Internal teams, low-risk
Namespace + Network Medium + NetworkPolicy + Quotas Teams with data concerns
vCluster Strong Virtual clusters Strong isolation, some sharing
Separate Clusters Strongest Separate control planes Compliance, high risk tenants

Most organizations end up between Tier 2 and Tier 3.

Tier 1: Namespace Isolation (The Minimum)

Even “just namespaces” should be done properly:

Namespace Labels for Pod Security

# Enforce restricted security policy on tenant namespaces
apiVersion: v1
kind: Namespace
metadata:
  name: team-alpha
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted
    tenant: team-alpha
    team: alpha

ResourceQuota — Non-Negotiable

apiVersion: v1
kind: ResourceQuota
metadata:
  name: team-alpha-quota
  namespace: team-alpha
spec:
  hard:
    # Compute
    requests.cpu: "20"
    requests.memory: 40Gi
    limits.cpu: "40"
    limits.memory: 80Gi
    # Storage
    requests.storage: 500Gi
    persistentvolumeclaims: "20"
    # Object counts
    pods: "50"
    services: "20"
    secrets: "50"
    configmaps: "50"
    # LoadBalancer services (expensive)
    services.loadbalancers: "2"
    services.nodeports: "0"  # No NodePort services

LimitRange — Force Container Limits

apiVersion: v1
kind: LimitRange
metadata:
  name: team-alpha-limits
  namespace: team-alpha
spec:
  limits:
  - type: Container
    default:
      cpu: "500m"
      memory: "512Mi"
    defaultRequest:
      cpu: "100m"
      memory: "128Mi"
    max:
      cpu: "4"
      memory: "8Gi"
    min:
      cpu: "50m"
      memory: "64Mi"
  - type: Pod
    max:
      cpu: "8"
      memory: "16Gi"
  - type: PersistentVolumeClaim
    max:
      storage: 50Gi

RBAC: Namespace Scoped

# Give team-alpha access only to their namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: team-alpha-developer
  namespace: team-alpha
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: edit  # Built-in edit role, scoped to namespace
subjects:
- kind: Group
  name: team-alpha-developers  # From your identity provider
  apiGroup: rbac.authorization.k8s.io
---
# Custom role for narrower permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: team-alpha-operator
  namespace: team-alpha
rules:
- apiGroups: [""]
  resources: ["pods", "services", "configmaps"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["apps"]
  resources: ["deployments", "statefulsets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list", "watch"]  # Read secrets, don't create/modify

Default Deny NetworkPolicy

# Every tenant namespace gets a default deny policy
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: team-alpha
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
---
# Allow DNS
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: team-alpha
spec:
  podSelector: {}
  egress:
  - ports:
    - port: 53
      protocol: UDP
    - port: 53
      protocol: TCP
---
# Allow intra-namespace communication
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-same-namespace
  namespace: team-alpha
spec:
  podSelector: {}
  ingress:
  - from:
    - podSelector: {}  # All pods in same namespace
  egress:
  - to:
    - podSelector: {}

Tier 2: Platform Engineering with HNC and Hierarchical Namespaces

Hierarchical Namespace Controller (HNC) lets you create namespace hierarchies where policies propagate automatically:

# Install HNC
kubectl apply -f https://github.com/kubernetes-sigs/hierarchical-namespaces/releases/latest/download/default.yaml
# Create a parent namespace for a tenant
apiVersion: v1
kind: Namespace
metadata:
  name: tenant-alpha
  annotations:
    hnc.x-k8s.io/managed-by: "hnc-controller"
---
# Child namespaces inherit from parent
apiVersion: hnc.x-k8s.io/v1alpha2
kind: SubnamespaceAnchor
metadata:
  name: team-alpha-backend
  namespace: tenant-alpha  # Parent namespace
---
apiVersion: hnc.x-k8s.io/v1alpha2
kind: SubnamespaceAnchor
metadata:
  name: team-alpha-frontend
  namespace: tenant-alpha

Policies in tenant-alpha propagate to all child namespaces automatically. This makes it easier to apply consistent policies across all of a tenant’s namespaces.

Tier 3: vCluster — Virtual Kubernetes Clusters

vCluster creates a fully functional Kubernetes API server inside a regular Kubernetes namespace. Tenants get their own API server, their own RBAC, and their own virtual cluster — while the underlying workloads still run on shared infrastructure:

# Install vCluster CLI
curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-linux-amd64"
sudo mv vcluster /usr/local/bin && chmod +x /usr/local/bin/vcluster

# Create a virtual cluster for team alpha
vcluster create team-alpha \
  --namespace team-alpha \
  --values vcluster-values.yaml
# vcluster-values.yaml
vcluster:
  # How the vCluster runs — using k3s for lightweight virtual control plane
  image: rancher/k3s:v1.28.2-k3s1

sync:
  # Sync these resource types between vCluster and host
  services:
    enabled: true
  configmaps:
    enabled: true
  secrets:
    enabled: true
  pods:
    enabled: true
    # Pods from the vCluster run in the host namespace with a prefix
    prefix: "team-alpha-"

# Resource limits for the vCluster control plane
resources:
  limits:
    cpu: 2
    memory: 4Gi

# The vCluster gets its own RBAC, its own namespaces
# Tenants interact only with the vCluster API — they never see the host cluster

Connect to a vCluster as the tenant:

# Get kubeconfig for the vCluster
vcluster connect team-alpha -n team-alpha --update-current=false

# Use it
KUBECONFIG=./kubeconfig-team-alpha.yaml kubectl get nodes
# The tenant sees virtual nodes, not real nodes

The vCluster approach gives tenants the full Kubernetes experience — they can create ClusterRoles, set up their own namespaces, install operators — while the host cluster maintains isolation through the vCluster boundary.

Automating Tenant Onboarding

Manual namespace setup is error-prone. Use a GitOps-based approach:

# Flux Kustomization that creates and configures a new tenant
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: tenant-team-alpha
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: cluster-config
  path: "./tenants/team-alpha"
  prune: true
  dependsOn:
  - name: infrastructure  # Wait for infrastructure components
# tenants/team-alpha/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- resourcequota.yaml
- limitrange.yaml
- rbac.yaml
- networkpolicy.yaml

This makes tenant onboarding a pull request, fully auditable, and reproducible.

Monitoring Across Tenants

Multi-tenant monitoring requires careful scoping:

# Give tenants access to their own metrics only
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant-metrics-reader
rules:
- apiGroups: ["metrics.k8s.io"]
  resources: ["pods", "nodes"]
  verbs: ["get", "list"]
  # Scope is enforced at the namespace level via RoleBinding
---
# In Prometheus, use namespace selectors to scope per-tenant dashboards
# Each team gets a Grafana dashboard scoped to their namespace(s)

Conclusion

Real Kubernetes multi-tenancy is a spectrum, not a binary. Namespaces with ResourceQuota, LimitRange, and NetworkPolicy get you surprisingly far for internal teams where the primary concern is resource contention and accidental interference.

For stronger isolation — where tenants are external customers, untrusted code runs, or regulatory separation is required — vCluster provides an excellent middle ground between “shared namespace” and “separate cluster.” It gives tenants a full Kubernetes experience while containing blast radius.

The pattern that works: define your isolation requirements first (what can’t one tenant do to another?), then choose the tier of isolation that addresses those requirements. Over-engineering with separate clusters for internal teams that trust each other adds operational overhead without proportional security benefit.

Start with namespace isolation done right, and migrate to vCluster when your isolation requirements demand it.

Share this post: LinkedIn Reddit WhatsApp Mastodon
Jesse Borden

Jesse Borden

Software Engineer with an interest in hands on learning

I have several years of professional Information Technology (IT) experience leading staff and projects within the Department of War (DOW). I have managed Service Desk, Web Application Development, and System Administration teams. My two greatest passions are learning and conti...