I have been using Helm or native kubectl YAML combined with CI and ArgoCD to deploy applications for a long time. Recently, while deploying private services for a major financial client, I found the client’s environment to be quite complex, involving multiple countries and multiple sets of environments, such as Test, UAT, and Prod—about thirty environments in total. Modifying values and YAML files every time was exhausting. This time, I plan to use Kustomize to refactor our existing application deployment.

Kustomize Introduction

What is Kustomize

Kustomize is a Kubernetes-native configuration management tool that manages YAML resources through a templating approach:

Core Features:

  • No template syntax, pure YAML
  • Declarative configuration management
  • Supports multi-environment management
  • GitOps friendly
  • Integrated into kubectl 1.14+

Comparison with Helm:

Feature Kustomize Helm
Template Syntax None (Pure YAML) Go Template
Learning Curve Low Medium-High
Multi-Environment Management Base + Overlay Values files
Package Management None Chart Repository
Version Management Git Chart Version
Suitable Scenarios Application Developer Application Distributor

Basic Concepts

kustomization.yaml
├── bases/          # Base configuration (reuse)
├── overlays/       # Environment overrides
│   ├── dev/
│   ├── staging/
│   └── prod/
├── patches/        # Configuration patches
└── resources/      # Static resources

Differences between Kustomize and Helm

Design Philosophy

Kustomize Design Philosophy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

# Declarative configuration management
# Does not use templates, but modifies configuration via Overlay

# Base: Base configuration
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 1
  template:
    spec:
      containers:
        - name: app
          image: my-app:latest

# Overlay: Environment differences
# overlays/production/kustomization.yaml
bases:
  - ../../base
patchesStrategicMerge:
  - |-
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: my-app
    spec:
      replicas: 10

Helm Design Philosophy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

# Templated configuration management
# Uses Go Template

# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Release.Name }}
spec:
  replicas: {{ .Values.replicaCount }}
  template:
    spec:
      containers:
        - name: app
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"

# values.yaml (Different values for different environments)
replicaCount: 1
image:
  repository: my-app
  tag: latest

# values-production.yaml
replicaCount: 10
image:
  repository: my-app
  tag: v1.2.3

Feature Comparison

Feature Kustomize Helm
Templates No templates, pure YAML Go Template
Learning Curve Low, only need to know YAML Medium-High, need to learn template syntax
Reusability Base + Overlay Chart + Values
Debugging kubectl kustomize helm template –dry-run
Version Management Git version control Chart version number
Environment Management Overlay directories Multiple Values files
Package Distribution None (use Git) Chart repository
CI/CD Integration Native kubectl Requires helm installation
Rollback kubectl rollout helm rollback
Dependency Management No direct support Chart dependencies

Applicable Scenarios

Scenarios for choosing Kustomize:

  • Application developers managing their own K8s resources
  • Need GitOps workflows
  • Large configuration differences across environments
  • Do not want to learn template languages
  • Team is familiar with YAML and kubectl

Scenarios for choosing Helm:

  • Application distributors (software vendors)
  • Need package management and version control
  • Relatively fixed configuration
  • Complex application dependencies
  • Quick deployment of third-party applications

Deploying K8s Applications with Kustomize

Project Structure

my-app/
├── base/                           # Base configuration
│   ├── deployment.yaml
│   ├── service.yaml
│   ├── configmap.yaml
│   ├── serviceaccount.yaml
│   └── kustomization.yaml
├── overlays/                       # Environment configuration
│   ├── dev/
│   │   ├── kustomization.yaml
│   │   ├── deployment-replica.yaml
│   │   ├── deployment-resources.yaml
│   │   └── configmap-env.yaml
│   ├── staging/
│   │   ├── kustomization.yaml
│   │   └── deployment-replica.yaml
│   └── production/
│       ├── kustomization.yaml
│       ├── deployment-replica.yaml
│       ├── deployment-resources.yaml
│       └── configmap-prod.yaml
└── scripts/
    └── deploy.sh

Creating Base Configuration

base/deployment.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  labels:
    app: my-app
    version: v1.0.0
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: my-app
        version: v1.0.0
    spec:
      serviceAccountName: my-app
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        fsGroup: 1000
      containers:
        - name: my-app
          image: my-app:latest
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          env:
            - name: JAVA_OPTS
              value: "-Xms512m -Xmx512m"
            - name: SPRING_PROFILES_ACTIVE
              value: "prod"
          resources:
            requests:
              cpu: 100m
              memory: 512Mi
            limits:
              cpu: 500m
              memory: 1Gi
          livenessProbe:
            httpGet:
              path: /actuator/health/liveness
              port: 8080
            initialDelaySeconds: 60
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /actuator/health/readiness
              port: 8080
            initialDelaySeconds: 30
            periodSeconds: 5
            timeoutSeconds: 3
            failureThreshold: 3
          volumeMounts:
            - name: config
              mountPath: /config
            - name: logs
              mountPath: /logs
      volumes:
        - name: config
          configMap:
            name: my-app-config
        - name: logs
          emptyDir: {}

base/service.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

apiVersion: v1
kind: Service
metadata:
  name: my-app
  labels:
    app: my-app
spec:
  type: ClusterIP
  ports:
    - port: 80
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app: my-app

base/configmap.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-app-config
data:
  application.yml: |
    server:
      port: 8080
    spring:
      application:
        name: my-app
    logging:
      level:
        root: INFO

base/serviceaccount.yaml

1
2
3
4

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app

base/kustomization.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# Referenced resource files
resources:
  - deployment.yaml
  - service.yaml
  - configmap.yaml
  - serviceaccount.yaml

# Common labels (added to all resources)
commonLabels:
  app.kubernetes.io/name: my-app
  app.kubernetes.io/instance: my-app
  app.kubernetes.io/managed-by: kustomize
  app.kubernetes.io/version: v1.0.0

# Common annotations
commonAnnotations:
  createdBy: kustomize
  env: base

# Name prefix
# namePrefix: dev-

# Name suffix
# nameSuffix: -v1

Overlay Environment Configuration

overlays/production/kustomization.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# Reference base
namespace: production
bases:
  - ../../base

# Replica count override
replicas:
  - name: my-app
    count: 10

# Image override
images:
  - name: my-app
    newName: harbor.example.com/prod/my-app
    newTag: v1.2.3

# ConfigMap generator
configMapGenerator:
  - name: my-app-config
    behavior: replace
    files:
      - application.yml=application-production.yml

# Secret generator
secretGenerator:
  - name: my-app-secret
    behavior: create
    envs:
      - .env.production

# Resource patches
patchesStrategicMerge:
  - deployment-resources.yaml
  - deployment-probes.yaml
  - pvc.yaml

# Namespace
namespace: production

# Common labels override
commonLabels:
  environment: production
  tier: backend

# Common annotations override
commonAnnotations:
  env: production
  prometheus.io/scrape: "true"
  prometheus.io/port: "8080"

overlays/production/deployment-resources.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      containers:
        - name: my-app
          resources:
            requests:
              cpu: 500m
              memory: 1Gi
            limits:
              cpu: 2000m
              memory: 4Gi

overlays/production/pvc.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-app-logs
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: fast-ssd
  resources:
    requests:
      storage: 50Gi

overlays/production/application-production.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

server:
  port: 8080

spring:
  application:
    name: my-app
  datasource:
    url: jdbc:mysql://mysql-prod.svc:3306/mydb
    username: prod_user
    password: ${DB_PASSWORD}

logging:
  level:
    root: INFO
    com.example: DEBUG
  file:
    path: /logs

overlays/production/.env.production

1
2
3

DB_PASSWORD=prod_secure_password_123
API_KEY=prod_api_key_xyz
REDIS_PASSWORD=prod_redis_pass

Deployment Commands

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

# Build configuration (do not apply, view final YAML)
kubectl kustomize overlays/production

# Save rendered result
kubectl kustomize overlays/production > rendered-prod.yaml

# View differences
kubectl diff -k overlays/production

# Apply directly
kubectl apply -k overlays/production

# Delete resources
kubectl delete -k overlays/production

# View resources
kubectl get all -l app.kubernetes.io/name=my-app -n production

Unified Deployment Management Configuration

1. Namespace Management

Method 1: Specify in Overlay

1
2

# overlays/production/kustomization.yaml
namespace: production

Method 2: Manage as Resource

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# overlays/production/kustomization.yaml
resources:
  - ../../base
  - namespace.yaml

# overlays/production/namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    environment: production
    name: production

2. Secret Management

Method 1: Environment Variable File

1
2
3
4
5
6

# overlays/production/kustomization.yaml
secretGenerator:
  - name: my-app-secret
    type: Opaque
    envs:
      - .env.production

Method 2: Literals

1
2
3
4
5
6

secretGenerator:
  - name: my-app-secret
    type: Opaque
    literals:
      - DB_PASSWORD=prod123
      - API_KEY=xyz789

Method 3: Command Generation

1
2
3
4
5
6

secretGenerator:
  - name: tls-cert
    type: kubernetes.io/tls
    files:
      - tls.crt=tls.crt
      - tls.key=tls.key

Method 4: Reference from Existing Secret

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# overlays/production/kustomization.yaml
generatorOptions:
  disableNameSuffixHash: true

resources:
  - secret.yaml

# overlays/production/secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: my-app-secret
type: Opaque
stringData:
  DB_PASSWORD: prod123

Method 5: Sealed Secrets (Recommended for Production)

1
2
3
4
5
6

# Generate sealed secret locally
kubeseal -f secret.yaml -w sealed-secret.yaml

# Reference in kustomization
resources:
  - sealed-secret.yaml

3. PVC Management

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

# overlays/production/kustomization.yaml
resources:
  - ../../base
  - pvc-logs.yaml
  - pvc-data.yaml

# overlays/production/pvc-logs.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-app-logs
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: fast-ssd  # Use SSD storage class
  resources:
    requests:
      storage: 50Gi

# overlays/production/pvc-data.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-app-data
spec:
  accessModes:
    - ReadWriteMany  # Multi-node shared
  storageClassName: nfs-storage
  resources:
    requests:
      storage: 100Gi

4. Image Version Management

Method 1: Direct Specification

1
2
3
4

images:
  - name: my-app
    newName: harbor.example.com/prod/my-app
    newTag: v1.2.3

Method 2: Use SHA256

1
2
3
4

images:
  - name: my-app
    newName: harbor.example.com/prod/my-app
    digest: sha256:abc123def456...  # Safest method

Method 3: Multiple Images

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

images:
  - name: nginx
    newName: harbor.example.com/library/nginx
    newTag: 1.21-alpine
  - name: redis
    newName: harbor.example.com/library/redis
    newTag: 6.2-alpine
  - name: my-app
    newName: harbor.example.com/prod/my-app
    newTag: v1.2.3

Method 4: CI/CD Dynamic Update

1
2
3
4
5

# In CI Pipeline
export IMAGE_TAG=$(git rev-parse --short HEAD)
cd overlays/production
kustomize edit set image my-app=harbor.example.com/prod/my-app:$IMAGE_TAG
kubectl apply -k .

5. Node Affinity Configuration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

# overlays/production/deployment-affinity.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      affinity:
        # Node affinity
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: node.kubernetes.io/instance-type
                    operator: In
                    values:
                      - c5.2xlarge
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              preference:
                matchExpressions:
                  - key: topology.kubernetes.io/zone
                    operator: In
                    values:
                      - us-west-1a
        # Pod affinity
        podAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                    - key: app
                      operator: In
                      values:
                        - mysql
                topologyKey: kubernetes.io/hostname

Reference in kustomization:

1
2

patchesStrategicMerge:
  - deployment-affinity.yaml

6. Taint and Toleration Configuration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

# overlays/production/deployment-taints.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      # Tolerate dedicated node taints
      tolerations:
        - key: workload
          operator: Equal
          value: dedicated
          effect: NoSchedule
        - key: node.kubernetes.io/not-ready
          operator: Exists
          effect: NoExecute
          tolerationSeconds: 300
        # Tolerate GPU nodes
        - key: nvidia.com/gpu
          operator: Exists
          effect: NoSchedule

7. Replica Count Management

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# Method 1: replicas field (Recommended)
replicas:
  - name: my-app
    count: 10
  - name: my-app-sidecar
    count: 2

# Method 2: Patch method
# overlays/production/deployment-replica.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 10

8. ConfigMap Management

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

# Method 1: configMapGenerator (Recommended)
configMapGenerator:
  # Generate from file
  - name: my-app-config
    behavior: replace
    files:
      - application.yml=application-production.yml
      - logback.xml=logback-production.xml

  # Generate from literals
  - name: my-app-env
    behavior: create
    literals:
      - SPRING_PROFILES_ACTIVE=production
      - LOG_LEVEL=INFO
      - JAVA_OPTS=-Xms2g -Xmx4g

  # Generate from env file
  - name: my-app-file-env
    behavior: create
    envs:
      - .env.production

# Method 2: Direct resource reference
resources:
  - configmap.yaml

Complete Production Environment Example

overlays/production/kustomization.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# Namespace
namespace: production

# Reference base
bases:
  - ../../base

# Additional resources
resources:
  - namespace.yaml
  - pvc-logs.yaml
  - pvc-data.yaml
  - hpa.yaml
  - service-monitor.yaml
  - network-policy.yaml

# Replica count
replicas:
  - name: my-app
    count: 10

# Image version
images:
  - name: my-app
    newName: harbor.example.com/prod/my-app
    newTag: v1.2.3

# ConfigMap
configMapGenerator:
  - name: my-app-config
    behavior: replace
    files:
      - application.yml=config/application-production.yml
  - name: my-app-env
    behavior: merge
    literals:
      - ENV=production
      - LOG_LEVEL=INFO

# Secret
secretGenerator:
  - name: my-app-secret
    behavior: create
    envs:
      - .env.production

# Patches
patchesStrategicMerge:
  - deployment-resources.yaml
  - deployment-affinity.yaml
  - deployment-taints.yaml
  - deployment-probes.yaml

# Common labels
commonLabels:
  environment: production
  tier: backend
  team: platform

# Common annotations
commonAnnotations:
  env: production
  prometheus.io/scrape: "true"
  prometheus.io/port: "8080"

CI/CD Integration

GitLab CI Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

# .gitlab-ci.yml
stages:
  - build
  - deploy

variables:
  REGISTRY: harbor.example.com
  APP_NAME: my-app

build:
  stage: build
  image: docker:24
  services:
    - docker:24-dind
  script:
    - docker build -t $REGISTRY/$APP_NAME:$CI_COMMIT_SHA .
    - docker push $REGISTRY/$APP_NAME:$CI_COMMIT_SHA
  only:
    - main
    - develop

deploy:dev:
  stage: deploy
  image: bitnami/kubectl:latest
  environment:
    name: dev
  script:
    - cd overlays/dev
    - kustomize edit set image $REGISTRY/$APP_NAME=$REGISTRY/$APP_NAME:$CI_COMMIT_SHA
    - kubectl apply -k .
    - kubectl rollout status deployment/my-app -n dev
  only:
    - develop

deploy:prod:
  stage: deploy
  image: bitnami/kubectl:latest
  environment:
    name: production
  script:
    - cd overlays/production
    - kustomize edit set image $REGISTRY/$APP_NAME=$REGISTRY/$APP_NAME:$CI_COMMIT_SHA
    - kubectl apply -k .
    - kubectl rollout status deployment/my-app -n production
  when: manual
  only:
    - main

ArgoCD Integration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

# argocd/application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app-production
  namespace: argocd
spec:
  project: production
  source:
    repoURL: https://github.com/example/k8s-manifests.git
    targetRevision: main
    path: overlays/production
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Summary

Kustomize Best Practices

  1. Directory Structure Standards

    • Store common configurations in base
    • Manage environment differences in overlays
    • Use clear naming conventions

  2. Version Management

    • Use Git to manage all configurations
    • Use SHA256 or explicit tags for images
    • Do not use latest

  3. Secret Management

    • Use Sealed Secrets in production environments
    • Do not commit sensitive information to Git
    • Use External Secrets Operator to integrate with Vault

  4. Component Reuse

    • Extract common configurations as components
    • Establish a Kustomize component library
    • Avoid duplicate configurations

  5. CI/CD Integration

    • Automate image tag updates
    • Use kubectl diff to preview changes
    • Apply configuration only after validation passes

Kustomize vs Helm Selection

Scenario Recommended Solution Reason
Microservice Configuration Management Kustomize High flexibility, Git-friendly
Third-party Application Deployment Helm Rich ecosystem, one-click deployment
Complex Template Requirements Helm Go Template is powerful
Simple Configuration Kustomize No need to learn templates
GitOps Kustomize Native support

Learning Path

  1. Basic Syntax: base + overlay structure
  2. Patch Strategies: StrategicMerge vs JSON6902
  3. Component Reuse: Extract common configurations
  4. CI/CD Integration: Automated deployment workflows
  5. GitOps Practice: Combine with ArgoCD/Flux

Kustomize is an ideal tool for configuration management in the cloud-native era. Mastering it can significantly improve Kubernetes application management efficiency. In actual deployment scenarios, if the differentiation between environments for new projects is small, you can combine VibeCoding tools to generate one-click deployment scripts for rapid delivery of new environments.