kubernetes-sigs/kustomize
1
0
Fork 0
mirror of https://github.com/kubernetes-sigs/kustomize.git synced 2026-05-17 18:25:26 +00:00
Code Issues Packages Projects Releases 1 Wiki Activity
Files
5fd7b14fc8f07b8f61dc710627d8ed0ce761495e
kustomize/proposals/22-03-value-in-the-structured-data.md
koba1t a34dd1fc3f fix replacements syntax
2023-12-17 05:48:56 +09:00

17 KiB
Raw Blame History

Replacements and Patch value in the structured data.

Authors:

  • koba1t

Reviewers:

  • natasha41575
  • knverey

Status: implementable

Summary

This proposal decides the interfaces to change values in the structured data (like json,yaml) inside a Kubernetes objects' field value and implements changing function target a few formats (mainly json).

Motivation

kustomize can apply structured edits to Kubernetes objects defined in yaml files.
Sometimes structured multi-line or long single line string (ex. json,yaml, and other structured format data) is injected in Kubernetes objects' string literal field. From the kustomize perspective, these "structured" multiline strings are just arbitrary unstructured strings.
So, kustomize can't manipulate one value on structured, formatted data in the Kubernetes object's string literal field. This function is expected behavior, but kustomize will be very helpful if it can change the value of structured data like json and yaml substrings in a string literal.
This proposal allows the user to identify strings literals containing JSON/YAML data so that Kustomize can make structured edits to the data they contain. This allows the requested functionality to be added without violating Kustomize's core principle of supporting structured edits exclusively.\

For example, kustomize can't change the value "REPLACE_TARGET_HOSTNAME" in this yaml file straightforwardly.

apiVersion: v1
kind: ConfigMap
metadata:
  name: target-configmap
data:
  config.json: |-
    {"config": {
      "id": "42",
      "hostname": "REPLACE_TARGET_HOSTNAME"
    }}

This function has been wanted for a long time.

This function can be an alternative deprecated vars function in many use cases.

Goals:

  1. Provide the way to update values in the structured data like kubernetes objects with JSON/YAML formats.

Non-goals:

  1. Do not provide Unstructured edits, because kustomize eschews parameterization.

  2. Do not provide a way for targeting/merging values within strings from the patches field.

  3. Do not provide a way for customizing field merge strategies for the embedded data.

Proposal

Replacement the value in structured data

I propose to extend the fieldPath and fieldPaths fields to replace the value in structured data with the replacements function.
This idea is extending how to select any field in replacement TargetSelector. If the source.fieldPath and targets.fieldPaths had extra values after a specific string literal in Yaml, Kustomize tries to parse that string as structure data and tries to drill down using that additional values.

Example.

## replacement
replacements:
- source:
    kind: ConfigMap
    name: source-configmap
    fieldPath: data.HOSTNAME
  targets:
  - select:
      kind: ConfigMap
      name: target-configmap
    fieldPaths:
    - data.config\.json.config.hostname # A path after `config\.json` is pointing one place in the structured data.

Please check Story 1.

Disciplined merge the value in structured data with configMapGenerator and secretGenerator

This Proposal is to add parameters for configMapGenerator and secretGenerator to allow the merging of two string literals when the behavior option is set to merge and string literals value is structured.
This idea is to add one parameter, mergeValues to GeneratorArgs used from configMapGenerator and secretGenerator. The mergeValues option is a list that contains two parameters, key and format. One element corresponds to one string literal, key is used to select a string literal to merge, and format is used to designate a format for string literal which is JSON or YAML.
This merge operation will be implemented for a part of Overriding Base ConfigMap Values. It will execute to merge two string literal having same key name when merging two configMap or secret.\

Example

configMapGenerator:
- name: demo-settings
  behavior: merge                     # This function requires `behavior: merge`.
  mergeValues:
  - key: config.json # Key with a target to merge.
    format: json # Setting structured data format MUST be YAML/JSON.
  literals:
  - config.json: |-
      {
        "config": {
          "hostname": "REPLACE_TARGET_HOSTNAME",
          "value": {
            "foo": "bar"
          }
        }
      }

Please check Story 2.

Appendix

User Stories

Story 1

Scenario summary: Replacement the value inside for structured data(json) in the configMap.

kustomize patching overlay is very strong to manage common yaml when using many cluster.
But, if you want to set cluster specific change value in the json with configMap data field, you have to replacement whole json file.

apiVersion: v1
kind: ConfigMap
metadata:
  name: target-configmap-dev
data:
  config.json: |-
    {"config": {
      "id": "42",
      "hostname": "dev.example.com
    }}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: target-configmap-prod
data:
  config.json: |-
    {"config": {
      "id": "42",
      "hostname": "prod.example.com"
    }}

So if we can replacement this value in the substring formatted with json, we can easy to overlay this difference.

## source
apiVersion: v1
kind: ConfigMap
metadata:
  name: source-configmap
data:
  HOSTNAME: www.example.com
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: target-configmap
data:
  config.json: |-
    {"config": {
      "id": "42",
      "hostname": "REPLACE_TARGET_HOSTNAME"
    }}

## replacement
replacements:
- source:
    kind: ConfigMap
    name: source-configmap
    fieldPath: data.HOSTNAME
  targets:
  - select:
      kind: ConfigMap
      name: target-configmap
    fieldPaths:
    - data.config\.json.config.hostname

## expected
apiVersion: v1
kind: ConfigMap
metadata:
  name: source-configmap
data:
  HOSTNAME: www.example.com
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: target-configmap
data:
  config.json: '{"config":{"hostname":"www.example.com","id":"42"}}'

Story 2

Scenario summary: Merge the two values formatted by structured data(json) with configMapGenerator.

Many application needs to be set with json format file. And, That file is handled with configMap when the application is running on kubernetes.
So, If kustomize configMapGenerator can merge for json inside a configMap data value, A json format file will be simple and easy to handle.

Source

# base/kustomization.yaml
configMapGenerator:
- name: demo
  literals:
  - config.json: |-
      {
        "config": {
          "loglevel": debug,
          "parameter": {
            "foo": "bar"
          }
        }
      }

# overlay/kustomization.yaml
resources:
- ../base
configMapGenerator:
- name: demo
  behavior: merge
  mergeValues:
  - key: config.json # Key with a target to merge.
    format: json # Setting structured data format MUST be YAML/JSON.
  literals:
  - config.json: |-
      {
        "config": {
          "hostname": "www.example.com",
          "parameter": {
            "baz": "qux"
          }
        }
      }

Result

apiVersion: v1
data:
  config.json: |-
    {
      "config": {
        "loglevel": debug,
        "hostname": "www.example.com",
        "parameter": {
          "foo": "bar",
          "baz": "qux"
        }
      }
    }
kind: ConfigMap
metadata:
  name: demo-xxxxxxxxxx  # name suffix hash

Story 3

Scenario summary: Replacement the value inside for yaml in the configMap.

A few applications require to use yaml format config file, and some major cloudnative applications are using that.(ex, Prometheus,AlertManager)
A value you want to overlay in yaml is usually into a nested yaml structure. So if you can overlay value inside yaml, you won't need to copy a whole yaml file.

## source
apiVersion: v1
kind: ConfigMap
metadata:
  name: environment-config
data:
  env: dev
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
data:
  prometheus.yml: |-
    global:
      external_labels:
        prometheus_env: TARGET_ENVIROMENT
    scrape_configs:
      - job_name: "prometheus"
        static_configs:
          - targets: ["localhost:9090"]

## replacement
replacements:
- source:
    kind: ConfigMap
    name: environment-config
    fieldPath: data.env
  targets:
  - select:
      kind: ConfigMap
      name: prometheus-config
    fieldPaths:
    - data.prometheus\.yml.global.external_labels?prometheus_env

## expected
apiVersion: v1
kind: ConfigMap
metadata:
  name: environment-config
data:
  env: dev
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
data:
  prometheus.yml: |-
    global:
      external_labels:
        prometheus_env: dev
    scrape_configs:
      - job_name: "prometheus"
        static_configs:
          - targets: ["localhost:9090"]

Story 4

Scenario summary: Replacement the value inside for json in the annotations.

A few times, an application on your cluster requires to set json format config for the Annotations on kubernetes yaml resources. We need to overlay in this position value.

Example
## source
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: debug-backend-config
spec:
  securityPolicy:
    name: "debug-security-policy"
---
apiVersion: v1
kind: Service
metadata:
  name: appA-svc
  annotations:
    cloud-provider/backend-config: '{"ports": {"appA":"gke-default-backend-config"}}'
spec:
  ports:
  - name: appA
    port: 1234
    protocol: TCP
    targetPort: 8080

## replacement
replacements:
- source:
    kind: BackendConfig
    name: debug-backend-config
    fieldPath: metadata.name
  targets:
  - select:
      kind: ConServicefigMap
      name: appA-svc
    fieldPaths:
    - metadata.annotations.cloud-provider/backend-config.ports.appA

## expected
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: debug-backend-config
spec:
  securityPolicy:
    name: "debug-security-policy"
---
apiVersion: v1
kind: Service
metadata:
  name: appA-svc
  annotations:
    cloud-provider/backend-config: '{"ports": {"appA":"debug-backend-config"}}'
spec:
  ports:
  - name: appA
    port: 1234
    protocol: TCP
    targetPort: 8080

Risks and Mitigations

Dependencies

Scalability

Drawbacks

Alternatives

Rollout Plan

Alpha

  • Will the feature be gated by an "alpha" flag? Which one?
  • Will the feature be available in kubectl kustomize during alpha? Why or why not?

Beta

GA

Reference in New Issue View Git Blame Copy Permalink