From 3276e74d2d37f2f33135e8d35e461a056e8a6ec2 Mon Sep 17 00:00:00 2001 From: Frank Farzan Date: Tue, 7 Jan 2020 11:44:31 -0800 Subject: [PATCH] Generalize configuration functions spec in RFC format. As defined in `kustomize config docs-fn-spec`, configuration functions can be implemented using any toolchain and invoked using any container workflow orchestrator (e.g. Tekton, Cloud Build) or run directly using docker run. functions-impl describes using `kustomize config run` as an example orchestrator for invoking configuration functions. --- cmd/config/configcobra/cmds.go | 9 +- cmd/config/docs/api-conventions/config-fn.md | 281 -------- .../docs/api-conventions/functions-impl.md | 181 +++++ .../docs/api-conventions/functions-spec.md | 186 ++++++ cmd/config/internal/generateddocs/api/docs.go | 631 ++++++++++-------- 5 files changed, 723 insertions(+), 565 deletions(-) delete mode 100644 cmd/config/docs/api-conventions/config-fn.md create mode 100644 cmd/config/docs/api-conventions/functions-impl.md create mode 100644 cmd/config/docs/api-conventions/functions-spec.md diff --git a/cmd/config/configcobra/cmds.go b/cmd/config/configcobra/cmds.go index 4129d2418..ebbf15a4a 100644 --- a/cmd/config/configcobra/cmds.go +++ b/cmd/config/configcobra/cmds.go @@ -93,8 +93,13 @@ func NewConfigCommand(name string) *cobra.Command { }) root.AddCommand(&cobra.Command{ Use: "docs-fn", - Short: "[Alpha] Documentation for writing containerized functions run by run.", - Long: api.ConfigFnLong, + Short: "[Alpha] Documentation for developing and invoking Configuration Functions.", + Long: api.FunctionsImplLong, + }) + root.AddCommand(&cobra.Command{ + Use: "docs-fn-spec", + Short: "[Alpha] Documentation for Configuration Functions Specification.", + Long: api.FunctionsSpecLong, }) root.AddCommand(&cobra.Command{ Use: "docs-io-annotations", diff --git a/cmd/config/docs/api-conventions/config-fn.md b/cmd/config/docs/api-conventions/config-fn.md deleted file mode 100644 index 6e4736330..000000000 --- a/cmd/config/docs/api-conventions/config-fn.md +++ /dev/null @@ -1,281 +0,0 @@ -# Configuration Functions API Semantics - - Configuration Functions are functions packaged as executables in containers which enable - **shift-left practices**. They configure applications and infrastructure through - Kubernetes style Resource Configuration, but run locally pre-commit. - - Configuration functions enable shift-left practices (client-side) through: - - - Pre-commit / delivery validation and linting of configuration - - e.g. Fail if any containers don't have PodSecurityPolicy or CPU / Memory limits - - Implementation of abstractions as client actuated APIs (e.g. templating) - - e.g. Create a client-side *"CRD"* for generating configuration checked into git - - Aspect Orient configuration / Injection of cross-cutting configuration - - e.g. T-Shirt size containers by annotating Resources with `small`, `medium`, `large` - and inject the cpu and memory resources into containers accordingly. - - e.g. Inject `init` and `side-car` containers into Resources based off of Resource - Type, annotations, etc. - - Performing these on the client rather than the server enables: - - - Configuration to be reviewed prior to being sent to the API server - - Configuration to be validated as part of the CD pipeline - - Configuration for Resources to validated holistically rather than individually - per-Resource -- e.g. ensure the `Service.selector` and `Deployment.spec.template` labels - match. - - MutatingWebHooks are scoped to a single Resource instance at a time. - - Low-level tweaks to the output of high-level abstractions -- e.g. add an `init container` - to a client *"CRD"* Resource after it was generated. - - Composition and layering of multiple functions together - - Compose generation, injection, validation together - - Configuration Functions are implemented as executable programs published in containers which: - - - Accept as input (stdin): - - A list of Resource Configuration - - A Function Configuration (to configure the function itself) - - Emit as output (stdout + exit): - - A list of Resource Configuration - - An exit code for success / failure - -### Function Specification - - - Functions **SHOULD** be published as container images containing a `CMD` invoking an executable. - - Functions **MUST** accept input on STDIN a `ResourceList` containing the Resources and - `functionConfig`. - - Functions **MUST** emit output on STDOUT a `ResourceList` containing the modified - Resources. - - Functions **MUST** exit non-0 on failure, and exit 0 on success. - - Functions **MAY** emit output on STDERR with error messaging. - - Functions performing validation **SHOULD** exit failure and emit error messaging - on a validation failure. - - Functions generating Resources **SHOULD** retain non-conflicting changes on the - generated Resources -- e.g. 1. the function generates a Deployment, but doesn't - specify `cpu`, 2. the user sets the `cpu` on the generated Resource, 3. the - function should keep the `cpu` when regenerating the Resource a second time. - - Functions **SHOULD** be usable outside `kustomize config run` -- e.g. though pipeline - mechanisms such as Tekton. - -#### Input Format - - Functions must accept on STDIN: - - `ResourceList`: - - contains `items` field, same as `List.items` - - contains `functionConfig` field -- a single item with the configuration for the function itself - - Example `ResourceList` Input: - - apiVersion: config.kubernetes.io/v1alpha1 - kind: ResourceList - functionConfig: - apiVersion: example.com/v1beta1 - kind: Nginx - metadata: - name: my-instance - annotations: - config.kubernetes.io/local-config: "true" - spec: - replicas: 5 - items: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: my-instance - spec: - replicas: 3 - ... - - apiVersion: v1 - kind: Service - metadata: - name: my-instance - spec: - ... - -#### Output Format - - Functions must emit on STDOUT: - - `ResourceList`: - - contains `items` field, same as `List.items` - - Example `ResourceList` Output: - - apiVersion: config.kubernetes.io/v1alpha1 - kind: ResourceList - items: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: my-instance - spec: - replicas: 5 - ... - - apiVersion: v1 - kind: Service - metadata: - name: my-instance - spec: - ... - -#### Container Environment - - When run by `kustomize config run`, functions are run in containers with the - following environment: - - - Network: `none` - - User: `nobody` - - Security Options: `no-new-privileges` - - Volumes: the volume containing the `functionConfig` yaml is mounted under `/local` as `ro` - -### Example Function Implementation - - Following is an example for implementing an nginx abstraction using a config - function. - -#### `nginx-template.sh` - - `nginx-template.sh` is a simple bash script which uses a *heredoc* as a templating solution - for generating Resources from the functionConfig input fields. - - The script wraps itself using `config run wrap -- $0` which will: - - 1. Parse the `ResourceList.functionConfig` (provided to the container stdin) into env vars - 2. Merge the stdout into the original list of Resources - 3. Defaults filenames for newly generated Resources (if they are not set as annotations) - to `config/NAME_KIND.yaml` - 4. Format the output - - #!/bin/bash - # script must run wrapped by `kustomize config run wrap` - # for parsing input the functionConfig into env vars - if [ -z ${WRAPPED} ]; then - export WRAPPED=true - config run wrap -- $0 - exit $? - fi - - cat <