kubernetes-sigs/kustomize
1
0
Fork 0
mirror of https://github.com/kubernetes-sigs/kustomize.git synced 2026-06-11 17:12:51 +00:00
Code Issues Packages Projects Releases 1 Wiki Activity

change kinflate to kustomize

Browse Source
This commit is contained in:
Jingfang Liu
2018-04-10 14:32:02 -07:00
committed by Sunil Arora
commit d5b5adfc5c
22 changed files with 1799 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
demos/README.md Normal file
View File

@@ -0,0 +1,13 @@
# Demos
These demos are covered by presubmit tests.
* [hello world](helloWorld.md) - Deploy multiple
(differently configured) instances of a simple Hello
World server.
* [mysql](mySql.md) - Create a mySql production
configuration from scratch.
* [springboot](springboot.md) - Create a Spring Boot application production
configuration from scratch.

432
demos/helloWorld.md Normal file
View File

@@ -0,0 +1,432 @@
[base]: ../docs/glossary.md#base
[config]: https://github.com/kinflate/example-hello
[hello]: https://github.com/monopole/hello
[instance]: ../docs/glossary.md#instance
[instances]: ../docs/glossary.md#instance
[manifest]: ../docs/glossary.md#manifest
[original]: https://github.com/kinflate/example-hello
[overlay]: ../docs/glossary.md#overlay
[overlays]: ../docs/glossary.md#overlay
# Demo: hello world with instances
Steps:
1. Clone an existing configuration as a [base].
1. Customize it.
1. Create two different [overlays] (_staging_ and _production_)
from the customized base.
1. Run kustomize and kubectl to deploy staging and production.
First define a place to work:
<!-- @makeWorkplace @test -->
```
DEMO_HOME=$(mktemp -d)
```
Alternatively, use
> ```
> DEMO_HOME=~/hello
> ```
## Clone an example
Let's run the [hello] service.
Here's an existing [config] for it.
Clone this config to a directory called `base`:
<!-- @cloneIt @test -->
```
git clone \
https://github.com/kinflate/example-hello \
$DEMO_HOME/base
```
<!-- @runTree @test -->
```
tree $DEMO_HOME
```
Expecting something like:
> ```
> /tmp/tmp.IyYQQlHaJP
> └── base
> ├── configMap.yaml
> ├── deployment.yaml
> ├── kustomize.yaml
> ├── LICENSE
> ├── README.md
> └── service.yaml
> ```
One could immediately apply these resources to a
cluster:
> ```
> kubectl apply -f $DEMO_HOME/base
> ```
to instantiate the _hello_ service. `kubectl`
would only recognize the resource files.
## The Base Manifest
The `base` directory has a [manifest] file:
<!-- @showManifest @test -->
```
BASE=$DEMO_HOME/base
more $BASE/kustomize.yaml
```
Run `kustomize` on the base to emit customized resources
to `stdout`:
<!-- @buildBase @test -->
```
kustomize build $BASE
```
## Customize the base
A first customization step could be to change the _app
label_ applied to all resources:
<!-- @addLabel @test -->
```
sed -i 's/app: hello/app: my-hello/' \
$BASE/kustomize.yaml
```
See the effect:
<!-- @checkLabel @test -->
```
kustomize build $BASE | grep -C 3 app:
```
## Create Overlays
Create a _staging_ and _production_ [overlay]:
* _Staging_ enables a risky feature not enabled in production.
* _Production_ has a higher replica count.
* Web server greetings from these cluster
[instances] will differ from each other.
<!-- @overlayDirectories @test -->
```
OVERLAYS=$DEMO_HOME/overlays
mkdir -p $OVERLAYS/staging
mkdir -p $OVERLAYS/production
```
#### Staging Manifest
In the `staging` directory, make a manifest
defining a new name prefix, and some different labels.
<!-- @makeStagingManifest @test -->
```
cat <<'EOF' >$OVERLAYS/staging/kustomize.yaml
apiVersion: manifest.k8s.io/v1alpha1
kind: Package
metadata:
name: makes-staging-hello
namePrefix: staging-
objectLabels:
instance: staging
org: acmeCorporation
objectAnnotations:
note: Hello, I am staging!
bases:
- ../../base
patches:
- map.yaml
EOF
```
#### Staging Patch
Add a configMap customization to change the server
greeting from _Good Morning!_ to _Have a pineapple!_
Also, enable the _risky_ flag.
<!-- @stagingMap @test -->
```
cat <<EOF >$OVERLAYS/staging/map.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: the-map
data:
altGreeting: "Have a pineapple!"
enableRisky: "true"
EOF
```
#### Production Manifest
In the production directory, make a manifest
with a different name prefix and labels.
<!-- @makeProductionManifest @test -->
```
cat <<EOF >$OVERLAYS/production/kustomize.yaml
apiVersion: manifest.k8s.io/v1alpha1
kind: Package
metadata:
name: makes-production-tuthello
namePrefix: production-
objectLabels:
instance: production
org: acmeCorporation
objectAnnotations:
note: Hello, I am production!
bases:
- ../../base
patches:
- deployment.yaml
EOF
```
#### Production Patch
Make a production patch that increases the replica
count (because production takes more traffic).
<!-- @productionDeployment @test -->
```
cat <<EOF >$OVERLAYS/production/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: the-deployment
spec:
replicas: 10
EOF
```
## Compare overlays
`DEMO_HOME` now contains:
- a _base_ directory - a slightly customized clone
of the original configuration, and
- an _overlays_ directory, containing the manifests
and patches required to create distinct _staging_
and _production_ instances in a cluster.
Review the directory structure and differences:
<!-- @listFiles @test -->
```
tree $DEMO_HOME
```
Expecting something like:
> ```
> /tmp/tmp.IyYQQlHaJP1
> ├── base
> │   ├── configMap.yaml
> │   ├── deployment.yaml
> │   ├── kustomize.yaml
> │   ├── LICENSE
> │   ├── README.md
> │   └── service.yaml
> └── overlays
> ├── production
> │   ├── deployment.yaml
> │   └── kustomize.yaml
> └── staging
> ├── kustomize.yaml
> └── map.yaml
> ```
Compare the output directly
to see how _staging_ and _production_ differ:
<!-- @compareOutput -->
```
diff \
<(kustomize build $OVERLAYS/staging) \
<(kustomize build $OVERLAYS/production) |\
more
```
The first part of the difference output should look
something like
> ```diff
> < altGreeting: Have a pineapple!
> < enableRisky: "true"
> ---
> > altGreeting: Good Morning!
> > enableRisky: "false"
> 8c8
> < note: Hello, I am staging!
> ---
> > note: Hello, I am production!
> 11c11
> < instance: staging
> ---
> > instance: production
> 13c13
> (...truncated)
> ```
## Deploy
The individual resource sets are:
<!-- @buildStaging @test -->
```
kustomize build $OVERLAYS/staging
```
<!-- @buildProduction @test -->
```
kustomize build $OVERLAYS/production
```
To deploy, pipe the above commands to kubectl apply:
> ```
> kustomize build $OVERLAYS/staging |\
> kubectl apply -f -
> ```
> ```
> kustomize build $OVERLAYS/production |\
> kubectl apply -f -
> ```
## Rolling updates
### Review
The _hello-world_ deployment running in this cluster is
configured with data from a configMap.
The deployment refers to this map by name:
<!-- @showDeployment @test -->
```
grep -C 2 configMapKeyRef $DEMO_HOME/base/deployment.yaml
```
Changing the data held by a live configMap in a cluster
is considered bad practice. Deployments have no means
to know that the configMaps they refer to have
changed, so such updates have no effect.
The recommended way to change a deployment's
configuration is to
1. create a new configMap with a new name,
1. patch the _deployment_, modifying the name value of
the appropriate `configMapKeyRef` field.
This latter change initiates rolling update to the pods
in the deployment. The older configMap, when no longer
referenced by any other resource, is eventually garbage
collected.
### How this works with kustomize
[patch]: ../docs/glossary.md#patch
The _staging_ instance here has a configMap [patch]:
<!-- @showMapPatch @test -->
```
cat $OVERLAYS/staging/map.yaml
```
This patch is by definition a named but not necessarily
complete resource spec intended to modify a complete
resource spec.
The resource it modifies is here:
<!-- @showMapBase @test -->
```
cat $DEMO_HOME/base/configMap.yaml
```
For a patch to work, the names in the `metadata/name`
fields must match.
However, the name values specified in the file are
_not_ what gets used in the cluster. By design,
kustomize modifies these names. To see the names
ultimately used in the cluster, just run kustomize:
<!-- @grepStagingName @test -->
```
kustomize build $OVERLAYS/staging |\
grep -B 8 -A 1 staging-the-map
```
The configMap name is prefixed by _staging-_, per the
`namePrefix` field in
`$OVERLAYS/staging/kustomize.yaml`.
The suffix to the configMap name is generated from a
hash of the maps content - in this case the name suffix
is _hhhhkfmgmk_:
<!-- @grepStagingHash @test -->
```
kustomize build $OVERLAYS/staging | grep hhhhkfmgmk
```
Now modify the map patch, to change the greeting
the server will use:
<!-- @changeMap @test -->
```
sed -i 's/pineapple/kiwi/' $OVERLAYS/staging/map.yaml
```
Run kustomize again to see the new names:
<!-- @grepStagingName @test -->
```
kustomize build $OVERLAYS/staging |\
grep -B 8 -A 1 staging-the-map
```
Confirm that the change in configMap content resulted
in three new names ending in _khk45ktkd9_ - one in the
configMap name itself, and two in the deployment that
uses the map:
<!-- @countHashes @test -->
```
test 3 == $(kustomize build $OVERLAYS/staging | grep khk45ktkd9 | wc -l)
```
Applying these resources to the cluster will result in
a rolling update of the deployments pods, retargetting
them from the _hhhhkfmgmk_ maps to the _khk45ktkd9_
maps. The system will later garbage collect the
unused maps.
## Rollback
To rollback, one would undo whatever edits were made to
the configuation in source control, then rerun kustomize
on the reverted configuration and apply it to the
cluster.

239
demos/mySql.md Normal file
View File

@@ -0,0 +1,239 @@
# Demo: MySql
This example takes some off-the-shelf k8s resources
designed for MySQL, and customizes them to suit a
production scenario.
In the production environment we want:
- MySQL resource names to be prefixed by 'prod-'.
- MySQL resources to have 'env: prod' labels.
- MySQL to use persistent disk for storing data.
### Download resources
Download `deployment.yaml`, `service.yaml` and
`secret.yaml`. These are plain k8s resources files one
could add to a k8s cluster to run MySql.
<!-- @makeMySQLDir @test -->
```
DEMO_HOME=$(mktemp -d)
cd $DEMO_HOME
# Get MySQL configs
for f in service secret deployment ; do \
wget https://raw.githubusercontent.com/kinflate/mysql/master/emptyDir/$f.yaml ; \
done
```
### Initialize a manifest
A _manifest_ groups these resources together.
Create one:
<!-- @initApp @test -->
```
cd $DEMO_HOME
kustomize init
```
You should now have a file called `kustomize.yaml`:
<!-- @catMan @test -->
```
cat $DEMO_HOME/kustomize.yaml
```
containing something like:
> ```
> apiVersion: manifest.k8s.io/v1alpha1
> kind: Manifest
> metadata:
> name: helloworld
> # description: helloworld does useful stuff.
> namePrefix: some-prefix
> # Labels to add to all objects and selectors.
> # These labels would also be used to form the selector for apply --prune
> # Named differently than “labels” to avoid confusion with metadata for this object
> objectLabels:
> app: helloworld
> objectAnnotations:
> note: This is a example annotation
> resources:
> - deployment.yaml
> - service.yaml
> # There could also be configmaps in Base, which would make these overlays
> configmaps: []
> # There could be secrets in Base, if just using a fork/rebase workflow
> secrets: []
> recursive: true
> ```
### Add the resources to the manifest
<!-- @addResources @test -->
```
cd $DEMO_HOME
kustomize edit add resource secret.yaml
kustomize edit add resource service.yaml
kustomize edit add resource deployment.yaml
cat kustomize.yaml
```
`kustomize.yaml`'s resources section should contain:
> ```
> apiVersion: manifest.k8s.io/v1alpha1
> ....
> resources:
> - secret.yaml
> - service.yaml
> - deployment.yaml
> ```
### Name Customization
Arrange for the MySQL resources to begin with prefix
_prod-_ (since they are meant for the _production_
environment):
<!-- @customizeLabel @test -->
```
cd $DEMO_HOME
kustomize edit set nameprefix 'prod-'
cat kustomize.yaml
```
`kustomize.yaml` should have updated value of namePrefix field:
> ```
> apiVersion: manifest.k8s.io/v1alpha1
> ....
> namePrefix: prod-
> objectAnnotations:
> note: This is a example annotation
> ```
This `namePrefix` directive adds _prod-_ to all
resource names.
<!-- @genNamePrefixConfig @test -->
```
kustomize build $DEMO_HOME
```
The output should contain:
> ```
> apiVersion: v1
> data:
> password: YWRtaW4=
> kind: Secret
> metadata:
> ....
> name: prod-mysql-pass-d2gtcm2t2k
> ---
> apiVersion: v1
> kind: Service
> metadata:
> ....
> name: prod-mysql
> spec:
> ....
> ---
> apiVersion: apps/v1beta2
> kind: Deployment
> metadata:
> ....
> name: prod-mysql
> spec:
> selector:
> ....
> ```
### Label Customization
We want resources in production environment to have
certain labels so that we can query them by label
selector.
`kustomize` does not have `set label` command to add
label, but we can edit `kustomize.yaml` file under
`prod` directory and add the production labels under
`objectLabels` fields as highlighted below.
<!-- @customizeLabels @test -->
```
sed -i 's/app: helloworld/app: prod/' \
$DEMO_HOME/kustomize.yaml
```
At this point, running `kustomize build` will
generate MySQL configs with name-prefix 'prod-' and
labels `env:prod`.
### Storage customization
Off the shelf MySQL uses `emptyDir` type volume, which
gets wiped away if the MySQL Pod is recreated, and that
is certainly not desirable for production
environment. So we want to use Persistent Disk in
production. kustomize lets you apply `patches` to the
resources.
<!-- @createPatchFile @test -->
```
cat <<'EOF' > $DEMO_HOME/persistent-disk.yaml
apiVersion: apps/v1beta2 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: mysql
spec:
template:
spec:
volumes:
- name: mysql-persistent-storage
emptyDir: null
gcePersistentDisk:
pdName: mysql-persistent-storage
EOF
```
Specify the patch file in the manifest:
<!-- @specifyPatch @test -->
```
cat <<'EOF' >> $DEMO_HOME/kustomize.yaml
patches:
- persistent-disk.yaml
EOF
```
Lets break this down:
- In the first step, we created a YAML file named
`persistent-disk.yaml` to patch the resource defined
in deployment.yaml
- Then we added `persistent-disk.yaml` to list of
`patches` in `kustomize.yaml`. `kustomize build`
will apply this patch to the deployment resource with
the name `mysql` as defined in the patch.
The output of the following command can now be applied
to the cluster (i.e. piped to `kubectl apply`) to
create the production environment.
<!-- @finalInflation @test -->
```
kustomize build $DEMO_HOME # | kubectl apply -f -
```

350
demos/springboot.md Normal file
View File

@@ -0,0 +1,350 @@
# Demo: SpringBoot
In this tutorial, you will learn - how to use `kustomize` to customize a basic Spring Boot application's
k8s configuration for production use cases.
In the production environment we want to customize the following:
- add application specific configuration for this Spring Boot application
- configure prod DB access configuration
- resource names to be prefixed by 'prod-'.
- resources to have 'env: prod' labels.
- JVM memory to be properly set.
- health check and readiness check.
### Download resources
Download `deployment.yaml`, `service.yaml`. These are plain k8s resources files one
could add to a k8s cluster to run sbdemo.
<!-- @makeSpringBootDir @test -->
```
DEMO_HOME=$(mktemp -d)
cd $DEMO_HOME
# Get SpringBoot configs
for f in service deployment; do \
wget https://raw.githubusercontent.com/kinflate/example-springboot/master/$f.yaml ; \
done
```
### Initialize a manifest
A _manifest_ groups these resources together.
Create one:
<!-- @initApp @test -->
```
cd $DEMO_HOME
kustomize init
```
The above step will create a `kustomize` configuration file called `kustomize.yaml` in current directory.
<!-- @catMan @test -->
```
cat $DEMO_HOME/kustomize.yaml
```
containing something like:
> ```
> apiVersion: manifest.k8s.io/v1alpha1
> kind: Manifest
> metadata:
> name: helloworld
> # description: helloworld does useful stuff.
> namePrefix: some-prefix
> # Labels to add to all objects and selectors.
> # These labels would also be used to form the selector for apply --prune
> # Named differently than “labels” to avoid confusion with metadata for this object
> objectLabels:
> app: helloworld
> objectAnnotations:
> note: This is a example annotation
> resources:
> - deployment.yaml
> - service.yaml
> # There could also be configmaps in Base, which would make these overlays
> configMapGenerator: []
> # There could be secrets in Base, if just using a fork/rebase workflow
> secretGenerator: []
> ```
### Add the resources to the manifest
<!-- @addResources @test -->
```
cd $DEMO_HOME
kustomize edit add resource service.yaml
kustomize edit add resource deployment.yaml
cat kustomize.yaml
```
`kustomize.yaml`'s resources section should contain:
> ```
> apiVersion: manifest.k8s.io/v1alpha1
> ....
> resources:
> - service.yaml
> - deployment.yaml
> ```
### Add configmap to the manifest
<!-- @addConfigMap @test -->
```
cd $DEMO_HOME
wget https://raw.githubusercontent.com/kinflate/example-springboot/master/application.properties
kustomize edit add configmap demo-configmap --from-file application.properties
cat kustomize.yaml
```
`kustomize.yaml`'s configMapGenerator section should contain:
> ```
> configMapGenerator:
> - files:
> - application.properties
> name: demo-configmap
> kind: Manifest
> ```
### Customize configmap
We want to add database credentials for the prod environment. In general, these credentials can be put into the file `application.properties`.
However, for some cases, we want to keep the credentials in a different file and keep application specific configs in `application.properties`.
With this clear separation, the credentials and application specific things can be managed and maintained flexibly by different teams.
For example, application developers only tune the application configs in `application.properties` and operation teams or SREs
only care about the credentials.
For Spring Boot application, we can set an active profile through the environment variable `spring.profiles.active`. Then
the application will pick up an extra `application-<profile>.properties` file. With this, we can customize the configmap in two
steps. Add an environment variable through the patch and add a file to the configmap.
<!-- @customizeConfigMap @test -->
```
cat <<EOF >$DEMO_HOME/patch.yaml
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: sbdemo
spec:
template:
spec:
containers:
- name: sbdemo
env:
- name: spring.profiles.active
value: prod
EOF
cat <<EOF >>$DEMO_HOME/kustomize.yaml
patches:
- patch.yaml
EOF
cat <<EOF >$DEMO_HOME/application-prod.properties
spring.jpa.hibernate.ddl-auto=update
spring.datasource.url=jdbc:mysql://<prod_database_host>:3306/db_example
spring.datasource.username=root
spring.datasource.password=admin
EOF
kustomize edit add configmap demo-configmap --from-file application-prod.properties
cat kustomize.yaml
```
`kustomize.yaml`'s configMapGenerator section should contain:
> ```
> configMapGenerator:
> - files:
> - application.properties
> - application-prod.properties
> name: demo-configmap
> kind: Manifest
> ```
### Name Customization
Arrange for the resources to begin with prefix
_prod-_ (since they are meant for the _production_
environment):
<!-- @customizeLabel @test -->
```
cd $DEMO_HOME
kustomize edit set nameprefix 'prod-'
cat kustomize.yaml
```
`kustomize.yaml` should have updated value of namePrefix field:
> ```
> apiVersion: manifest.k8s.io/v1alpha1
> ....
> namePrefix: prod-
> objectAnnotations:
> note: This is a example annotation
> ```
This `namePrefix` directive adds _prod-_ to all
resource names.
<!-- @genNamePrefixConfig @test -->
```
kustomize build $DEMO_HOME
```
The output should contain:
> ```
> apiVersion: v1
> data:
> ....
> kind: ConfigMap
> metadata:
> ....
> name: prod-demo-configmap-7746248cmc
> ---
> apiVersion: v1
> kind: Service
> metadata:
> ....
> name: prod-sbdemo
> spec:
> ....
> ---
> apiVersion: apps/v1beta2
> kind: Deployment
> metadata:
> ....
> name: prod-sbdemo
> spec:
> selector:
> ....
> ```
### Label Customization
We want resources in production environment to have
certain labels so that we can query them by label
selector.
`kustomize` does not have `edit set label` command to add
label, but we can edit `kustomize.yaml` file under
`prod` directory and add the production labels under
`objectLabels` fields as highlighted below.
<!-- @customizeLabels @test -->
```
sed -i 's/app: helloworld/app: prod/' \
$DEMO_HOME/kustomize.yaml
```
At this point, running `kustomize build` will
generate MySQL configs with name-prefix 'prod-' and
labels `env:prod`.
### Download Patch for JVM memory
When a Spring Boot application is deployed in a k8s cluster, the JVM is running inside a container. We want to set memory limit for the container and make sure
the JVM is aware of that limit. In K8s deployment, we can set the resource limits for containers and inject these limits to
some environment variables by downward API. When the container starts to run, it can pick up the environment variables and
set JVM options accordingly.
Download the patch `memorylimit_patch.yaml`. It contains the memory limits setup.
<!-- @downloadPatch @test -->
```
cd $DEMO_HOME
wget https://raw.githubusercontent.com/kinflate/example-springboot-instances/master/production/memorylimit_patch.yaml
cat memorylimit_patch.yaml
```
The output contains
> ```
> apiVersion: apps/v1beta2
> kind: Deployment
> metadata:
> name: sbdemo
> spec:
> template:
> spec:
> containers:
> - name: sbdemo
> resources:
> limits:
> memory: 1250Mi
> requests:
> memory: 1250Mi
> env:
> - name: MEM_TOTAL_MB
> valueFrom:
> resourceFieldRef:
> resource: limits.memory
> ```
### Download Patch for health check
We also want to add liveness check and readiness check in the production environment. Spring Boot application
has end points such as `/actuator/health` for this. We can customize the k8s deployment resource to talk to Spring Boot end point.
Download the patch `healthcheck_patch.yaml`. It contains the liveness probes and readyness probes.
<!-- @downloadPatch @test -->
```
cd $DEMO_HOME
wget https://raw.githubusercontent.com/kinflate/example-springboot-instances/master/production/healthcheck_patch.yaml
cat healthcheck_patch.yaml
```
The output contains
> ```
> apiVersion: apps/v1beta2
> kind: Deployment
> metadata:
> name: sbdemo
> spec:
> template:
> spec:
> containers:
> - name: sbdemo
> livenessProbe:
> httpGet:
> path: /actuator/health
> port: 8080
> initialDelaySeconds: 10
> periodSeconds: 3
> readinessProbe:
> initialDelaySeconds: 20
> periodSeconds: 10
> httpGet:
> path: /actuator/info
> port: 8080
> ```
### Add patch to Manifest
Currently `kustomize` doesn't provide a command to add a file as a patch, but we can edit the file `kustomize.yaml` to
include this patch.
<!-- @addPatch @test -->
```
mv $DEMO_HOME/kustomize.yaml $DEMO_HOME/tmp.yaml
sed '/patches:$/{N;s/- patch.yaml/- patch.yaml\n- memorylimit_patch.yaml\n- healthcheck_patch.yaml/}' $DEMO_HOME/tmp.yaml >& $DEMO_HOME/kustomize.yaml
```
`kustomize.yaml` should have patches field:
> ```
> patches
> - patch.yaml
> - memorylimit_patch.yaml
> - healthcheck_patch.yaml
> ```
The output of the following command can now be applied
to the cluster (i.e. piped to `kubectl apply`) to
create the production environment.
<!-- @finalBuild @test -->
```
kustomize build $DEMO_HOME # | kubectl apply -f -
```

BIN
demos/tree1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.5 KiB

BIN
demos/tree2.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 14 KiB