diff --git a/docs/plugins.md b/docs/plugins.md index fe0ca45a9..86ab59048 100644 --- a/docs/plugins.md +++ b/docs/plugins.md @@ -1,20 +1,36 @@ # kustomize plugins -Kustomize offers a plugin framework for people to -write their own resource _generators_ (e.g. a helm -chart processor, a generator that automatically -attaches a Service and Ingress object to each -Deployment) and their own resource _transformers_ -(e.g. a transformer that does some highly -customized processing of the container command -line). +Kustomize offers a plugin framework allowing +people to write their own resource _generators_ +and _transformers_. + +[generator options]: ../examples/generatorOptions.md +[transformer configs]: ../examples/transformerconfigs + +Write a plugin when changing [generator options] +or [transformer configs] doesn't meet your needs. + +[12-factor]: https://12factor.net + + * A _generator_ plugin could be a helm chart + inflator, or a plugin that emits all the + components (deployment, service, scaler, + ingress, etc.) needed by someone's [12-factor] + application, based on a smaller number of free + variables. + + * A _transformer_ plugin might perform special + container command line edits, or any other + transformation that exceeds the power of the + builtin transformations (`namePrefix`, + `commonLabels`, etc.). ## Specification in `kustomization.yaml` Start by adding a `generators` and/or `transformers` field to your kustomization. -Each field is a string array: +Each field accepts a string list: > ``` > generators: @@ -27,11 +43,13 @@ Each field is a string array: > - {as above} > ``` -This is exactly like the syntax of the `resources` field. +This is exactly like the syntax of the `resources` +field. -The value of each entry in a `resources`, `generators` -or `transformers` array must be a relative path to a -YAML file, or a path or URL to a [kustomization]. +The value of each entry in a `resources`, +`generators` or `transformers` list must be a +relative path to a YAML file, or a path or URL +to a [kustomization]. [kustomization]: glossary.md#kustomization @@ -39,9 +57,9 @@ In the former case the YAML is read from disk directly, and in the latter case a kustomization is performed, and its YAML output is merged with the YAML read directly from files. The net result in all three cases -is an array of YAML objects. +is a set of YAML objects. -_Each_ object resulting from a `generators` or +Each object resulting from a `generators` or `transformers` field is now further interpreted by kustomize as a _plugin configuration_ object. @@ -58,6 +76,8 @@ Given this, the kustomization process would expect to find a file called `chartInflator.yaml` in the kustomization [root](glossary.md#kustomization-root). +This is the _plugin's configuration file_. + The file `chartInflator.yaml` could contain: ``` @@ -68,50 +88,51 @@ metadata: chartName: minecraft ``` -The `apiVersion` and `kind` fields of the configuration -objects are used to _locate_ the plugin. +__The `apiVersion` and `kind` fields are +used to locate the plugin.__ -The rest of the file (actually the entire file) is -sent to the plugin as configuration - i.e. as the -plugin's construction arguments. +[k8s object]: glossary.md#kubernetes-style-object -A kustomization file could include multiple -instantiations of the same plugin, with different -arguments (e.g. to inflate two different helm -charts or two instances of the same chart but with -different values files). +> Thus, these fields are required. They are also +> required because a kustomize plugin +> configuration object is also a [k8s object]. -The value order in the `generators` field doesn't -matter, because generated objects are just added -to a sea of objects that kustomize transforms and -emits. +To get the plugin ready to generator or transform, +it is given the entire contents of the +configuration file. -The specified order of transformers in the -`transformers` field should be respected, as -transformers cannot be expected to be commutative. +[NameTransformer]: ../plugin/builtin/nametransformer/NameTransformer_test.go +[ChartInflator]: ../plugin/someteam.example.com/v1/chartinflator/ChartInflator_test.go +[plugins]: ../plugin/builtin -## Execution - -Plugins are only used during a run of the -`kustomize build` command. - -Generator plugins are run after processing the -`resources` field (which _reads_ resources), to -_create_ additional resources. - -The full set of resources is then passed into the -transformation pipeline, where native (legacy) -transformations like `namePrefix` and -`commonLabel` are applied, followed by all the -transformers run in the order specified. +For more examples of plugin configuration YAML, +browse the unit tests below the [plugins] root, +e.g. the tests for [ChartInflator] or +[NameTransformer]. ## Placement -[k8s object]: glossary.md#kubernetes-style-object +Each plugin gets its own dedicated directory named -Given a plugin configuration object (it looks like any -other [k8s object]), kustomize will first look for an +``` +$XDG_CONFIG_HOME/kustomize/plugin + /${apiVersion}/LOWERCASE(${kind}) +``` + +The default value of `XDG_CONFIG_HOME` is +`$HOME/.config`. + +The one-plugin-per-directory requirement eases +creation of a plugin tarball (source, test, plugin +data files, etc.) for sharing. + +In the case of a [Go plugin](#go-plugins), it also +allows one to provide a `go.mod` file for the +single plugin, easing resolution of package +version dependency skew. + +When loading, kustomize will first look for an _executable_ file called ``` @@ -119,24 +140,45 @@ $XDG_CONFIG_HOME/kustomize/plugin /${apiVersion}/LOWERCASE(${kind})/${kind} ``` -The default value of `XDG_CONFIG_HOME` is `$HOME/.config`. - If this file is not found or is not executable, kustomize will look for a file called `${kind}.so` in the same directory and attempt to load it as a [Go plugin](#go-plugins). -If both checks fails, the plugin load fails the overall -kustomize build. +If both checks fail, the plugin load fails the overall +`kustomize build`. -A `kustomize build` attempt with plugins that +## Execution + +Plugins are only used during a run of the +`kustomize build` command. + +Generator plugins are run after processing the +`resources` field (which itself is in some sense a +generator in that it emits resources for further +processing). + +The full set of resources is then passed into the +transformation pipeline, wherein builtin +transformations like `namePrefix` and +`commonLabel` are applied (if they were specified +in the kustomization file), followed by the +user-specified transformers in the `transformers` +field. + +The specified order of transformers in the +`transformers` field should be respected, as +transformers cannot be expected to be commutative. + +A `kustomize build` that tries to use plugins but omits the flag +_TODO: Change flag_ + > `--enable_alpha_goplugins_accept_panic_risk` will fail with a warning about plugin use. -_TODO: Change flag_ Flag use is an opt-in acknowledging the absence of plugin provenance. It's meant to give pause to @@ -147,34 +189,44 @@ code in plugin form. The plugin would have to be installed already, but nevertheless the flag is a reminder. + ## Writing plugins ### Exec plugins -[chartinflator]: ../plugin/someteam.example.com/v1/chartinflator/ChartInflator - -See this example [helm chart inflator][chartInflator]. - -A exec plugin is any executable that accepts a +A _exec plugin_ is any executable that accepts a single argument on its command line - the name of -a YAML file containing its configuration. +a YAML file containing its configuration (the file name +provided in the kustomization file). > TODO: more restrictions on plugin to allow the same exec > plugin to be specified in a config under both the > `generators` and `transformers` fields. > - first arg could be the fixed string > `generate` or `transform`, -> (the name of the configuration file moves to +> (the name of the configuration file moves to > the 2nd arg), or > - by default an exec plugin behaves as a tranformer > unless a flag `-g` is provided, switching the > exec plugin to behave as a generator. +[helm chart inflator]: ../plugin/someteam.example.com/v1/chartinflator +[bashed config map]: ../plugin/someteam.example.com/v1/bashedconfigmap +[sed transformer]: ../plugin/someteam.example.com/v1/sedtransformer + +#### Examples + + * [helm chart inflator] - A generator that inflates a helm chart. + * [bashed config map] - Super simple configMap generation from bash. + * [sed transformer] - Define your unstructured edits using a + plugin like this one. + + A generator plugin accepts nothing on `stdin`, but emits generated resources to `stdout`. A transformer plugin accepts resource YAML on `stdin`, -and emits those resources, possibly transformed, to +and emits those resources, presumably transformed, to `stdout`. kustomize uses an exec plugin adapter to provide @@ -184,9 +236,6 @@ marshalled resources on `stdin` and capture ### Go plugins [Go plugin]: https://golang.org/pkg/plugin/ -[servicegenerator]: ../plugin/someteam.example.com/v1/someservicegenerator/SomeServiceGenerator.go - -See this example [service generator][serviceGenerator]. A [Go plugin] for kustomize looks like this: @@ -213,60 +262,90 @@ A [Go plugin] for kustomize looks like this: > func (p *plugin) Transform(m resmap.ResMap) error {...} > ``` -The use of the identifiers `plugin`, -`KustomizePlugin` and the three method signatures -`Configurable`, `Generator`, `Transformer` as -shown is _required_. +Use of the identifiers `plugin`, `KustomizePlugin` +and implementation of the method signature +`Config` is required. -The plugin author will change the -contents of the `plugin` struct, and the three -method bodies, and add imports as desired. +Implementing the `Generator` or `Transformer` +method allows (respectively) the plugin's config +file to be added to the `generators` or +`transformers` field in the kustomization file. +Do one or the other or both as desired. -Here's a build command, which assumes the plugin -source code is sitting right next to where the -shared object (`.so`) files are expected to be: +[secret generator]: ../plugin/someteam.example.com/v1/secretsfromdatabase +[service generator]: ../plugin/someteam.example.com/v1/someservicegenerator +[string prefixer]: ../plugin/someteam.example.com/v1/stringprefixer +[date prefixer]: ../plugin/someteam.example.com/v1/dateprefixer + + +#### Examples + + * [secret generator] - Generate secrets from a database. + * [service generator] - Generate a service from a name and port argument. + * [string prefixer] - uses the value in `metadata/name` as the prefix. + This particular example exists to show how a plugin can + transform the behavior of a plugin. See the + `TestTransformedTransformers` test in the `target` package. + * [date prefixer] - prefix the current date to resource names, a simple + example used to modify the string prefixer plugin just mentioned. + * All the builtin plugins [here](../plugin/builtin). + User authored plugins are + on the same footing as builtin operations. + +A plugin can be both a generator and a +transformer. The `Generate` method will run along +with all the other generators before the +`Transform` method runs. + +Here's a build command that sensibly assumes the +plugin source code sits in the directory where +kustomize expects to find `.so` files: ``` -d=$XDG_CONFIG_HOME/kustomize/plugin/${apiVersion}/LOWERCASE(${kind}) -go build -buildmode plugin -o $d/${kind}.so $d/${kind}.go +d=$XDG_CONFIG_HOME/kustomize/plugin\ +/${apiVersion}/LOWERCASE(${kind}) + +go build -buildmode plugin \ + -o $d/${kind}.so $d/${kind}.go ``` #### Caveats -Go plugins allow kustomize extensions that - - * can be tested with the same framework kustomize - uses to test its _builtin_ generators and - transformers, - - * run without the performance cost of firing up a - subprocess and marshalling/unmarshalling all - resource data for each plugin run. +Go plugins allow kustomize extensions that run +without the cost marshalling/unmarshalling all +resource data to/from a subprocess for each plugin +run. [ELF]: https://en.wikipedia.org/wiki/Executable_and_Linkable_Format Go plugins work as [defined][Go plugin], but fall -short of what many people think of when they hear -the word _plugin_. Go plugin compilation creates -an [ELF] formatted `.so` file, which by definition -has no information about the _provenance_ of the -file. +short of common notions associated with the word +_plugin_. Go plugin compilation creates an [ELF] +formatted `.so` file, which by definition has no +information about the provenance of the object +code. Skew between the compilation conditions +(versions of package dependencies, `GOOS`, +`GOARCH`) of the main program ELF and the plugin +ELF will cause plugin load failure. -One cannot know which version of Go was used, -which packages were imported (and their version), -what value of `GOOS` and `GOARCH` were used, -etc. Skew between the compilation conditions of -the main program ELF and the plugin ELF will cause -a failure at load time. +Exec plugins also lack provenance, but won't +complain about compilation skew. -Exec plugins also lack provenance, but don't -suffer from the skew problem. +In either case, a sensible way to share a plugin +is as a tar file of source code, tests and +associated data, unpackable under +`$XDG_CONFIG_HOME/kustomize/plugin` (exactly where +one would develop a plugin). -In either case, at the time of writing the proper -way to share a plugin is as a tar file of source code -and associated data, developed and unpacked under -`kustomize/plugin`. In the case of a Go plugin, the -end user must compile it (described above), and may -need to compile kustomize as well. If people use -Go plugins, more tooling will be built to make -plugin sharing easier. +[Go modules]: https://github.com/golang/go/wiki/Modules + +In the case of a Go plugin, an end user accepting +a shared plugin must compile both kustomize and +the plugin. Tooling could be built to make Go +_plugin sharing_ easier, but this requires some +critical mass of _plugin authoring_, which in turn +is hampered by confusion around sharing. +[Go modules], once they are more widely adopted, +will solve one of the biggest plugin sharing +difficulties - ambiguous plugin vs host +dependencies. diff --git a/examples/README.md b/examples/README.md index 35fc7e61d..61ac72776 100644 --- a/examples/README.md +++ b/examples/README.md @@ -16,7 +16,7 @@ go get sigs.k8s.io/kustomize * [last mile helm](chart.md) - Make last mile modifications to a helm chart. - + * [LDAP](ldap/README.md) - Deploy multiple (differently configured) variants of a LDAP server. @@ -29,28 +29,27 @@ go get sigs.k8s.io/kustomize * [combineConfigs](combineConfigs.md) - Mixing configuration data from different owners (e.g. devops/SRE and developers). - + * [configGenerations](configGeneration.md) - Rolling update when ConfigMapGenerator changes. - * [secret generation](kvSourceGoPlugin.md) - Generating secrets. - + * [secret generation](secretGeneratorPlugin.md) - Generating secrets from a plugin. + * [generatorOptions](generatorOptions.md) - Modifying behavior of all ConfigMap and Secret generators. * [breakfast](breakfast.md) - Customize breakfast for Alice and Bob. - + * [vars](wordpress/README.md) - Injecting k8s runtime data into container arguments (e.g. to point wordpress to a SQL service) by vars. - + * [image names and tags](image.md) - Updating image names and tags without applying a patch. * [multibases](multibases/README.md) - Composing three variants (dev, staging, production) with a common base. * [remote target](remoteBuild.md) - Building a kustomization from a github URL - + * [json patch](jsonpatch.md) - Apply a json patch in a kustomization - + * [transformer configs](transformerconfigs/README.md) - Customize transformer configurations - diff --git a/examples/kvSourceGoPlugin.md b/examples/kvSourceGoPlugin.md deleted file mode 100644 index d5870687f..000000000 --- a/examples/kvSourceGoPlugin.md +++ /dev/null @@ -1,378 +0,0 @@ -[ConfigMaps]: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#configmap-v1-core -[ELF]: https://en.wikipedia.org/wiki/Executable_and_Linkable_Format -[Go plugin]: https://golang.org/pkg/plugin -[Secrets]: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#secret-v1-core -[base64]: https://tools.ietf.org/html/rfc4648#section-4 -[configuration directory]: https://wiki.archlinux.org/index.php/XDG_Base_Directory#Specification -[grpc]: https://grpc.io -[tag]: https://github.com/kubernetes-sigs/kustomize/releases -[v2.0.3]: https://github.com/kubernetes-sigs/kustomize/releases/tag/v2.0.3 -[`exec.Command`]: https://golang.org/pkg/os/exec/#Command - -# Generating Secrets - -## What's a Secret? - -Kubernetes [ConfigMaps] and [Secrets] are both -key:value (KV) maps, but the latter is intended to -signal that its values have a sensitive nature - -e.g. ssh keys or passwords. - -Kubernetes assumes that the values in a Secret are -[base64] encoded, and decodes them before actual -use (as, say, the argument to a container -command). The user that creates the Secret must -base64 encode the data, or use a tool that does it -for them. This encoding doesn't protect the -secret from anything other than an -over-the-shoulder glance. - -Protecting the actual secrecy of a Secret value is -up to the cluster operator. They must lock down -the cluster (and its `etcd` data store) as tightly -as desired, and likewise protect the bytes that -feed into the cluster to ultimately become the -content of a Secret value. - -## Make a place to work - - -``` -DEMO_HOME=$(mktemp -d) -``` - -## Secret values from local files - -kustomize has three different ways to generate a secret -from local files: - - * get them from so-called _env_ files (`NAME=VALUE`, one per line), - * consume the entire contents of a file to make one secret value, - * get literal values from the kustomization file itself. - -Here's an example combining all three methods: - -Make an env file with some short secrets: - - -``` -cat <<'EOF' >$DEMO_HOME/foo.env -ROUTER_PASSWORD=admin -DB_PASSWORD=iloveyou -EOF -``` - -Make a text file with a long secret: - - -``` -cat <<'EOF' >$DEMO_HOME/longsecret.txt -Lorem ipsum dolor sit amet, -consectetur adipiscing elit, -sed do eiusmod tempor incididunt -ut labore et dolore magna aliqua. -EOF -``` - -And make a kustomization file referring to the -above and additionally defining some literal KV -pairs: - - -``` -cat <<'EOF' >$DEMO_HOME/kustomization.yaml -secretGenerator: -- name: mysecrets - kvSources: - - name: envfiles - pluginType: builtin - args: - - foo.env - - name: files - pluginType: builtin - args: - - longsecret.txt - - name: literals - pluginType: builtin - args: - - FRUIT=apple - - VEGETABLE=carrot -EOF -``` - -> The above syntax is _alpha_ behavior at HEAD, for v2.1+. -> -> The default value of `pluginType` is `builtin`, so the -> `pluginType` fields could be omitted. -> -> The equivalent [v2.0.3] syntax (still supported) is -> ``` -> secretGenerator: -> - name: mysecrets -> env: foo.env -> files: -> - longsecret.txt -> literals: -> - FRUIT=apple -> - VEGETABLE=carrot -> ``` - -Now generate the Secret: - - -``` -result=$(kustomize build $DEMO_HOME) -echo "$result" -# Spot check the result: -test 1 == $(echo "$result" | grep -c "FRUIT: YXBwbGU=") -``` - -This emits something like - -> ``` -> apiVersion: v1 -> kind: Secret -> metadata: -> name: mysecrets-hfb5df789h -> type: Opaque -> data: -> FRUIT: YXBwbGU= -> VEGETABLE: Y2Fycm90 -> ROUTER_PASSWORD: YWRtaW4= -> DB_PASSWORD: aWxvdmV5b3U= -> longsecret.txt: TG9yZW0gaXBzdW0gZG9sb3Igc2l0I... (elided) -> ``` - -The name of the resource is a prefix, `mysecrets` -(as specfied in the kustomization file), followed -by a hash of its contents. - -Use your favorite base64 decoder to confirm the raw -versions of any of these values. - -The problem that these three approaches share is -that the purported secrets must live on disk. - -This adds additional security questions - who can -see the files, who installs them, who deletes -them, etc. - - -## Secret values from anywhere - -> New _alpha_ behavior at HEAD, for v2.1+ - -A general alternative is to enshrine secret -value generation in a [Go plugin]. - -The values can then come in via, say, an -authenticated and authorized RPC to a password -vault service. - -Here's a trivial plugin that provides -hardcoded values: - - -``` -cat <<'EOF' >$DEMO_HOME/kvMaker.go -package main -var database = map[string]string{ - "TREE": "oak", - "ROCKET": "Saturn V", - "FRUIT": "apple", - "VEGETABLE": "carrot", - "SIMPSON": "homer", -} - -type plugin struct{} -var KVSource plugin -func (p plugin) Get( - root string, args []string) (map[string]string, error) { - r := make(map[string]string) - for _, k := range args { - v, ok := database[k] - if ok { - r[k] = v - } - } - return r, nil -} -EOF -``` - - -The two crucial items needed to -load and query the plugin are - 1) the public symbol `KVSource`, - 1) its public `Get` method and signature. - -Plugins that generate KV pairs for kustomize -must be installed at - -> ``` -> $XDG_CONFIG_HOME/kustomize/plugin/kvSource -> ``` - -`XDG_CONFIG_HOME` is an environment variable -honored by many programs as the root of a -[configuration directory]. If the variable is -undefined, the convention is to fall back to -`$HOME/.config`. - -The rest of the required directory path -establishes that the files found there are -kustomize plugins for generating KV pairs. - -Compile and install the plugin: - - -``` -kvSources=$DEMO_HOME/kustomize/plugin/kvSources -mkdir -p $kvSources -go build -buildmode plugin \ - -o $kvSources/kvMaker.so \ - $DEMO_HOME/kvMaker.go -``` - -Create a new kustomization file -referencing this plugin: - - -``` -cat <<'EOF' >$DEMO_HOME/kustomization.yaml -secretGenerator: -- name: mysecrets - kvSources: - - name: kvMaker - pluginType: go - args: - - FRUIT - - VEGETABLE -EOF -``` - -Finally, generate the secret, setting -`XDG_CONFIG_HOME` so that the plugin -can be found under `$DEMO_HOME`: - - -``` -result=$( \ - XDG_CONFIG_HOME=$DEMO_HOME \ - kustomize \ - --enable_alpha_goplugins_accept_panic_risk \ - build $DEMO_HOME ) -echo "$result" -# Spot check the result: -test 1 == $(echo "$result" | grep -c "FRUIT: YXBwbGU=") -``` - -This should emit something like: - -> ``` -> apiVersion: v1 -> kind: Secret -> metadata: -> name: mysecrets-bdt27dbkd6 -> type: Opaque -> data: -> FRUIT: YXBwbGU= -> VEGETABLE: Y2Fycm90 -> ``` - -i.e. a subset of the same values as above. - -### Go Plugin Caveats - -Kustomize supports Go plugins to allow someone to -extend kustomize in type-safe fashion against a -documented Go interface type, without having to -get their code merged into the kustomize -repository, and without having to maintain a -permanent fork. - -Go plugins work well, but fall short of what many -people think of when they hear the word _plugin_. - -Go plugin compilation creates an [ELF] formatted -`.so` file, which by definition has no information -about the _provenance_ of the file. One cannot -know which version of Go was used, which packages -were imported (and their version), what value of -`GOOS` and `GOARCH` were used, etc. If the skew -between the compilation conditions of the main -program ELF and the plugin ELF are too great, the -program will crash. Also, there's no certificate -to check in a `.so` file, so no way to know who -wrote it or what it does. A bare `.so` file, not -packaged with provenance information, is not a -suitable distrubution format. It's not what -people expect from decades of adding features -IDEs, browsers, CAD tools, graphics tools, etc. -via things called _plugins_. - -There's no reason why someone couldn't build a -`.so` packaging mechanism into `go` to emit an ELF -packaged with provenance allowing ELF -compatibility checks, but this isn't supported in -kustomize (or Go) at the time of writing. - -To avoid provenance issues simply compile your Go -plugins and the main program at the same time. -Bundle them into a container image for use by -downstream users and/or your continuous delivery -bot. This is the intended usage idiom for Go -plugins. - -A `kustomize build` attempt with Go plugins that omits -the flag - -> `--enable_alpha_goplugins_accept_panic_risk` - -will fail with an error message about skew risks. -Flag use is an opt-in acknowledging the absence of -`.so` provenance, an absence that doesn't matter -to someone building the code from source. - - -### Leveraging Go plugins to run non-Go code - - -#### external services - -For particular (user-created) transformations or -generations, kustomize could prepare a request, -send it to some service, and process a response. -How to do this is a [solved problem][grpc]. The -communication is struct-to-struct type safe - no -need to write parsing code. - -If the service is written in Go, and one can -vendor its code, it's simplest to write a small Go -plugin that calls it like a library rather than -running the service as an independent process. - -If the service is not written in Go, or if the -source code is unavailable, one can use a small Go -plugin to make the RPC. - - -#### subprocesses (also known as `exec` plugins) - -In this approach one arranges for executable files -to be identified by name or location, and runs -them as a kustomize subprocess, sending a -'request' to the subprocess its `stdin`, and -obtaining a 'response' via its `stdout`. - -An immediate way to use an arbitrary executable -with arbitrary i/o requirements is through a Go -plugin that runs the executable via -[`exec.Command`]. Each special purpose -tranformation or generation - needed by `kustomize -build` - will require it's own `stdin`/`stdout` -processing to convert from/to the Go types that -kustomize uses. - -The Go plugin provides this translation layer, and -handles process exit codes. diff --git a/examples/secretGeneratorPlugin.md b/examples/secretGeneratorPlugin.md new file mode 100644 index 000000000..eee9ab42b --- /dev/null +++ b/examples/secretGeneratorPlugin.md @@ -0,0 +1,225 @@ +[ConfigMaps]: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#configmap-v1-core +[ELF]: https://en.wikipedia.org/wiki/Executable_and_Linkable_Format +[Go plugin]: https://golang.org/pkg/plugin +[Secrets]: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#secret-v1-core +[base64]: https://tools.ietf.org/html/rfc4648#section-4 +[configuration directory]: https://wiki.archlinux.org/index.php/XDG_Base_Directory#Specification +[grpc]: https://grpc.io +[tag]: https://github.com/kubernetes-sigs/kustomize/releases +[v2.0.3]: https://github.com/kubernetes-sigs/kustomize/releases/tag/v2.0.3 +[`exec.Command`]: https://golang.org/pkg/os/exec/#Command + +# Generating Secrets + +## What's a Secret? + +Kubernetes [ConfigMaps] and [Secrets] are both +key:value maps, but the latter is intended to +signal that its values have a sensitive nature - +e.g. pass phrases or ssh keys. + +Kubernetes developers work in various ways to hide +the information in a Secret more carefully than +the information held by ConfigMaps, Deployments, +etc. + +## Make a place to work + + +``` +DEMO_HOME=$(mktemp -d) +``` + +## Secret values from local files + +kustomize has three different (builtin) ways to +generate a secret from local files: + + * get them from so-called _env_ files (`NAME=VALUE`, one per line), + * consume the entire contents of a file to make one secret value, + * get literal values from the kustomization file itself. + +Here's an example combining all three methods: + +Make an env file with some short secrets: + + +``` +cat <<'EOF' >$DEMO_HOME/foo.env +ROUTER_PASSWORD=admin +DB_PASSWORD=iloveyou +EOF +``` + +Make a text file with a long secret: + + +``` +cat <<'EOF' >$DEMO_HOME/longsecret.txt +Lorem ipsum dolor sit amet, +consectetur adipiscing elit, +sed do eiusmod tempor incididunt +ut labore et dolore magna aliqua. +EOF +``` + +And make a kustomization file referring to the +above and _additionally_ defining some literal KV +pairs in line: + + +``` +cat <<'EOF' >$DEMO_HOME/kustomization.yaml +secretGenerator: +- name: mysecrets + envs: + - foo.env + files: + - longsecret.txt + literals: + - FRUIT=apple + - VEGETABLE=carrot +EOF +``` + +Now generate the Secret: + + +``` +result=$(kustomize build $DEMO_HOME) +echo "$result" +# Spot check the result: +test 1 == $(echo "$result" | grep -c "FRUIT: YXBwbGU=") +``` + +This emits something like + +> ``` +> apiVersion: v1 +> kind: Secret +> metadata: +> name: mysecrets-hfb5df789h +> type: Opaque +> data: +> FRUIT: YXBwbGU= +> VEGETABLE: Y2Fycm90 +> ROUTER_PASSWORD: YWRtaW4= +> DB_PASSWORD: aWxvdmV5b3U= +> longsecret.txt: TG9yZW0gaXBzdW0gZG9sb3Igc2l0I... (elided) +> ``` + +The name of the resource is the prefix `mysecrets` +(as specfied in the kustomization file), followed +by a hash of its contents. + +Use your favorite base64 decoder to confirm the raw +versions of any of these values. + +The problem that these three approaches share is +that the purported secrets must live on disk. + +This adds additional security questions - who can +see the files, who installs them, who deletes +them, etc. + + +## Secret values from anywhere + +> New _alpha_ behavior at HEAD, for v2.1+ + +A general alternative is to enshrine secret +value generation in a [plugin](../docs/plugins.md). + +The values can then come in via, say, an +authenticated and authorized RPC to a password +vault service. + +[sgp]: ../plugin/someteam.example.com/v1/secretsfromdatabase + +Here's a [secret generator plugin][sgp] +that pretends to pull the values of a map +from a database. + + +Download it + + +``` +repo=https://raw.githubusercontent.com/kubernetes-sigs/kustomize +pPath=plugin/someteam.example.com/v1/secretsfromdatabase +dir=$DEMO_HOME/kustomize/$pPath + +mkdir -p $dir + +curl -s -o $dir/SecretsFromDatabase.go \ + ${repo}/master/$pPath/SecretsFromDatabase.go +``` + +Compile it + + +``` +go build -buildmode plugin \ + -o $dir/SecretsFromDatabase.so \ + $dir/SecretsFromDatabase.go +``` + + +Create a configuration file for it: + + +``` +cat <<'EOF' >$DEMO_HOME/secretFromDb.yaml +apiVersion: someteam.example.com/v1 +kind: SecretsFromDatabase +metadata: + name: mySecretGenerator +name: forbiddenValues +namespace: production +keys: +- ROCKET +- VEGETABLE +EOF +``` + +Create a new kustomization file +referencing this plugin: + + +``` +cat <<'EOF' >$DEMO_HOME/kustomization.yaml +generators: +- secretFromDb.yaml +EOF +``` + +Finally, generate the secret, setting +`XDG_CONFIG_HOME` so that the plugin +can be found under `$DEMO_HOME`: + + +``` +result=$( \ + XDG_CONFIG_HOME=$DEMO_HOME \ + kustomize \ + --enable_alpha_goplugins_accept_panic_risk \ + build $DEMO_HOME ) +echo "$result" +# Spot check the result: +test 1 == $(echo "$result" | grep -c "FRUIT: YXBwbGU=") +``` + +This should emit something like: + +> ``` +> apiVersion: v1 +> kind: Secret +> metadata: +> name: mysecrets-bdt27dbkd6 +> type: Opaque +> data: +> FRUIT: YXBwbGU= +> VEGETABLE: Y2Fycm90 +> ``` + +i.e. a subset of the same values as above. diff --git a/plugin/doc.go b/plugin/doc.go index eafb4081c..40f9da59d 100644 --- a/plugin/doc.go +++ b/plugin/doc.go @@ -2,11 +2,17 @@ // SPDX-License-Identifier: Apache-2.0 /* + Package plugin contains builtin and example plugins, tests and test libraries, and a code generator for converting a plugin to statically loadable code (see pluginator). +See ../../docs/plugins.md for a description of +writing and testing a plugin. The information +here is supplemental to that, and more oriented to +how the builting plugins work. + HOW PLUGINS RUN @@ -25,21 +31,21 @@ Assume a file 'secGen.yaml' containing If this file were referenced by a kustomization file in its 'generators' field, kustomize would - * Read 'secGen.yaml'. +* Read 'secGen.yaml'. - * Use the value of $XGD_CONFIG_HOME and - 'apiversion' and to find an executable - named 'SecretGenerator' to use as - an exec plugin, or failing that, +* Use the value of $XGD_CONFIG_HOME and + 'apiversion' and to find an executable + named 'SecretGenerator' to use as + an exec plugin, or failing that, - * use the same info to load a Go plugin - object file called 'SecretGenerator.so'. +* use the same info to load a Go plugin + object file called 'SecretGenerator.so'. - * Send either the file name 'secGen.yaml' as - the first arg to the exec plugin, or send its - contents to the go plugin's Config method. +* Send either the file name 'secGen.yaml' as + the first arg to the exec plugin, or send its + contents to the go plugin's Config method. - * Use the plugin to generate and/or transform. +* Use the plugin to generate and/or transform. GO PLUGINS @@ -50,41 +56,19 @@ which useful functions are attached. It can further be used as a _kustomize_ plugin if the symbol is named 'KustomizePlugin' and the -attached functions implement the Configurable, -Generator and Transformer interfaces. +attached functions implement the `Configurable`, +`Generator` and `Transformer` interfaces. -A plugin won't load into some program foo/main.go +A plugin won't load into some program `foo/main.go` if there is any package version mismatch in the dependencies of the plugin and the dependencies of foo/main.go. Control this with matching -declarations in go.mod files. The versions of the +declarations in `go.mod` files. The versions of the builtin packages "fmt", "io", "os" (not normally -listed in go.mod) etc have the same version as the +listed in `go.mod`) etc have the same version as the compiler. -ONE PLUGIN PER DIRECTORY - -For kustomize (and perhaps anyone), it's simplest -to put each plugin into its own directory. - -Go plugins must be in package `main`, and so -having more than one plugin in a directory means -their loading symbols have to differ, which makes -it hard to standardize around how they get loaded, -or it means one must use build tags to suppress -full directory compilation - which creates -difficulties using IDEs, the `go mod` tool, `go -test ./...`, etc. - -A one plugin per directory policy makes it easy to -define the plugin as a module, with its own -`go.mod` file - which is vital for resolving -package version dependency mismatches at load -time. It also makes it easy to create a plugin -tarball (source, test, go.mod, plugin data files, -etc.) for distribution. - BUILTIN PLUGIN CONFIGURATION @@ -96,7 +80,6 @@ follows. The plugin config file looks like - --------------------------------------------- apiVersion: builtin kind: SecretGenerator metadata: @@ -104,20 +87,14 @@ The plugin config file looks like otherField1: whatever otherField2: whatever ... - --------------------------------------------- -The apiVersion must be 'builtin'. The kind is the -CamelCase name of the plugin. +The apiVersion must be 'builtin'. +The kind is the CamelCase name of the plugin. -For non-builtins the apiVersion can be any legal -apiVersion value, e.g. 'someteam.example.com/v1beta1' - -The builtin source must be at: +The source for a builtin plugin must be at: repo=$GOPATH/src/sigs.k8s.io/kustomize - ${repo}/plugin/${apiVersion}/LOWERCASE(${kind})/${kind}.go - -(dropping the ".go" for exec plugins). + ${repo}/plugin/builtin/LOWERCASE(${kind})/${kind} k8s wants 'kind' values to follow CamelCase, while Go style doesn't like but does allow such names. @@ -129,36 +106,9 @@ optional associated files (possibly a go.mod file). PLUGIN SOURCE - * Pattern - - secretgenerator.go - --------------------------------------------- - //go:generate go run sigs.k8s.io/kustomize/cmd/pluginator - package main - import ... - type plugin struct{...} - var KustomizePlugin plugin - func (p *plugin) Config( - ldr ifc.Loader, - rf *resmap.Factory, - c []byte) error {...} - func (p *plugin) Generate( - ) (resmap.ResMap, error) {...} - func (p *plugin) Transform( - m resmap.ResMap) error {...} - --------------------------------------------- - - The plugin name doesn't appear in the file itself. - - * Compilation - - repo=$GOPATH/src/sigs.k8s.io/kustomize - dir=$repo/plugin/builtin - go build -buildmode plugin \ - -o $dir/secretgenerator.so \ - $dir/secretgenerator.go - - + See ../../docs/plugins.md + for a description of writing and testing + a plugin. BUILTIN PLUGIN GENERATION @@ -169,43 +119,40 @@ code. It arises from following requirements: - * extension +* extension + kustomize does two things - generate or + transform k8s resources. Plugins let + users write their own G&T's without + having to fork kustomize and learn its + internals. - kustomize does two things - generate or - transform k8s resources. Plugins let - users write their own G&T's without - having to fork kustomize and learn its - internals. +* dogfooding + A G&T extension framework one can trust + should be used by its authors to deliver + builtin G&T's. - * dogfooding - - A G&T extension framework one can trust - should be used by its authors to deliver - builtin G&T's. - - * distribution - - kustomize should be distributable via - `go get` and should run where Go +* distribution + kustomize should be distributable via + `go get` and should run where Go programs are expected to run. - The extension requirement led to the creation - of a framework that accommodates writing a - G or T as either +The extension requirement led to the creation +of a framework that accommodates writing a +G or T as either - * an 'exec' plugin (any executable file - runnable as a kustomize subprocess), or +* an 'exec' plugin (any executable file + runnable as a kustomize subprocess), or - * as a Go plugin - see - https://golang.org/pkg/plugin. +* as a Go plugin - see + https://golang.org/pkg/plugin. - The dogfooding (and an implicit performance - requirement) requires a 'builtin' G or T to - be written as a Go plugin. +The dogfooding (and an implicit performance +requirement) requires a 'builtin' G or T to +be written as a Go plugin. - The distribution ('go get') requirement demands - conversion of Go plugins to statically linked - code, hence this program. +The distribution ('go get') requirement demands +conversion of Go plugins to statically linked +code, hence this program. TO GENERATE CODE @@ -222,7 +169,6 @@ etc. Generated plugins are used in kustomize via - --------------------------------------------- package whatever import "sigs.k8s.io/kustomize/plugin/builtin ... @@ -231,7 +177,6 @@ Generated plugins are used in kustomize via resources, err := g.Generate() err = g.Transform(resources) // Eventually emit resources. - --------------------------------------------- */ package plugin diff --git a/plugin/someteam.example.com/v1/configmapgenerator/ConfigMapGenerator b/plugin/someteam.example.com/v1/bashedconfigmap/BashedConfigMap similarity index 100% rename from plugin/someteam.example.com/v1/configmapgenerator/ConfigMapGenerator rename to plugin/someteam.example.com/v1/bashedconfigmap/BashedConfigMap diff --git a/plugin/someteam.example.com/v1/configmapgenerator/ConfigMapGenerator_test.go b/plugin/someteam.example.com/v1/bashedconfigmap/BashedConfigMap_test.go similarity index 83% rename from plugin/someteam.example.com/v1/configmapgenerator/ConfigMapGenerator_test.go rename to plugin/someteam.example.com/v1/bashedconfigmap/BashedConfigMap_test.go index d9ea5fd74..486cada80 100644 --- a/plugin/someteam.example.com/v1/configmapgenerator/ConfigMapGenerator_test.go +++ b/plugin/someteam.example.com/v1/bashedconfigmap/BashedConfigMap_test.go @@ -10,18 +10,18 @@ import ( "sigs.k8s.io/kustomize/plugin" ) -func TestConfigMapGeneratorPlugin(t *testing.T) { +func TestBashedConfigMapPlugin(t *testing.T) { tc := plugin.NewEnvForTest(t).Set() defer tc.Reset() tc.BuildExecPlugin( - "someteam.example.com", "v1", "ConfigMapGenerator") + "someteam.example.com", "v1", "BashedConfigMap") th := kusttest_test.NewKustTestPluginHarness(t, "/app") m := th.LoadAndRunGenerator(` apiVersion: someteam.example.com/v1 -kind: ConfigMapGenerator +kind: BashedConfigMap metadata: name: whatever argsOneLiner: alice myMomsMaidenName diff --git a/plugin/someteam.example.com/v1/secretsfromdatabase/SecretsFromDatabase.go b/plugin/someteam.example.com/v1/secretsfromdatabase/SecretsFromDatabase.go index d3c08587f..38b12e43d 100644 --- a/plugin/someteam.example.com/v1/secretsfromdatabase/SecretsFromDatabase.go +++ b/plugin/someteam.example.com/v1/secretsfromdatabase/SecretsFromDatabase.go @@ -50,5 +50,4 @@ func (p *plugin) Generate() (resmap.ResMap, error) { } } return p.rf.FromSecretArgs(p.ldr, nil, args) - }