doc/glossary updates for v2.1

docs/README.md

@@ -8,6 +8,8 @@
    [kustomization](glossary.md#kustomization)
    with explanations of each field.
 
+ * [plugins](plugins.md) Extending kustomize with custom generators and transformers.
+ 
  * [versioning policy](versioningPolicy.md) - How the code and the kustomization
    file evolve in time.
 

docs/glossary.md

@@ -31,6 +31,7 @@
 [rebase]: https://git-scm.com/docs/git-rebase
 [resource]: #resource
 [resources]: #resource
+[root]: #kustomization-root
 [rpm]: https://en.wikipedia.org/wiki/Rpm_(software)
 [strategic-merge]: https://github.com/kubernetes/community/blob/master/contributors/devel/strategic-merge-patch.md
 [target]: #target
@@ -74,10 +75,10 @@ management in k8s.
 
 ## base
 
-A _base_ is a [target] that some [overlay] modifies.
+A _base_ is a [kustomization] that some [overlay] modifies.
 
-Any target, including an [overlay], can be a base to
+Any kustomization, including an [overlay], can be a base to
-another target.
+another kustomization.
 
 A base has no knowledge of the overlays that refer to it.
 
@@ -142,31 +143,105 @@ test or deploy) when that truth changes.
 
 ## kustomization
 
-A _kustomization_ is a file called `kustomization.yaml` that
+The term _kustomization_ refers to a
-describes a configuration consumable by [kustomize].
+`kustomization.yaml` file, or more generally to a
+directory (the [root]) containing the
+`kustomization.yaml` file and all the relative file
+paths that it immediately references (all the local
+data that doesn't require a URL specification).
+
+I.e. if someone gives you a _kustomization_ for use
+with [kustomize], it could be in the form of
+
+ * one file called `kustomization.yaml`,
+ * a tarball (containing that YAML file plus what it references),
+ * a git archive (ditto),
+ * a URL to a git repo (ditto), etc.
+
+Here's an [example](kustomization.yaml) `kustomization.yaml`.
+
+A kustomization file contains fields falling into four
+categories:
+
+ * _resources_ - what existing [resources] are to be customized.
+   Example fields: _resources_, _crds_.
+
+ * _generators_ - what _new_ resources should be created.
+   Example fields: _configMapGenerator_ (legacy), 
+   _secretGenerator_ (legacy), _generators_ (v2.1).
+   
+ * _transformers_ - what to _do_ to the aforementioned resources.
+   Example fields: _namePrefix_, _nameSuffix_, _images_,
+   _commonLabels_, _patchesJson6902_, etc. and the more
+   general _transformers_ (v2.1) field.
+
+ * _meta_ - fields which may influence all or some of
+   the above.  Example fields: _vars_, _namespace_,
+   _apiVersion_, _kind_, etc.
 
 
-Here's an [example](kustomization.yaml).
+## kustomization root
 
-A kustomization contains fields falling into these categories:
+The directory that immediately contains a
+`kustomization.yaml` file.
 
- * _Customization operators_ for modifying operands, e.g.
+When a kustomization file is processed, it may or may
-   _namePrefix_, _nameSuffix_, _commonLabels_, _patches_, etc.
+not be able to access files outside its root.
 
- * _Customization operands_:
+Data files like resource YAML files, or text files
-   * [resources] - completely specified k8s API objects,
+containing _name=value_ pairs intended for a ConfigMap
-      e.g. `deployment.yaml`, `configmap.yaml`, etc.
+or Secret, or files representing a patch to be used in
-   * [bases] - paths or github URLs specifying directories
+a patch transformation, must live _within or below_ the
-     containing a [kustomization].  These bases may
+root, and as such are specified via _relative
-     be subjected to more customization, or merely
+paths_ exclusively.
-     included in the output.
-   * [CRD]s - custom resource definition files, to allow use
-     of _custom_ resources in the _resources_ list.
-     Not an actual operand - but allows the use of new operands.
 
- * Generators, for creating more resources
+A special flag (in v2.1), `--load_restrictions none`,
-   (configmaps and secrets) which can then be
+is provided to relax this security feature, to, say,
-   customized.
+allow a patch file to be shared by more than one
+kustomization.
+
+Other kustomizations (other directories containing a
+`kustomization.yaml` file) may be referred to by URL, by
+absolute path, or by relative path.
+
+If kustomization __A__ depends on kustomization __B__, then
+
+ * __B__ may not _contain_ __A__.
+ * __B__ may not _depend on_ __A__, even transitively.
+
+__A__ may contain __B__, but in this case it might be
+simplest to have __A__ directly depend on __B__'s
+resources and eliminate __B__'s kustomization.yaml file
+(i.e. absorb __B__ into __A__).
+
+Conventionally, __B__ is in a directory that's sibling
+to __A__, or __B__ is off in a completely independent
+git repository, referencable from any kustomization.
+
+
+A common layout is
+
+> ```
+> ├── base
+> │   ├── deployment.yaml
+> │   ├── kustomization.yaml
+> │   └── service.yaml
+> └── overlays
+>     ├── dev
+>     │   ├── kustomization.yaml
+>     │   └── patch.yaml
+>     ├── prod
+>     │   ├── kustomization.yaml
+>     │   └── patch.yaml
+>     └── staging
+>         ├── kustomization.yaml
+>         └── patch.yaml
+> ```
+
+The three roots `dev`, `prod` and `staging`
+(presumably) all refer to the `base` root.  One would
+have to inspect the `kustomization.yaml` files to be
+sure.
 
 ## kubernetes
 
@@ -189,14 +264,14 @@ more than one version).
 
 ## kustomize
 
-_kustomize_ is a command line tool supporting template-free
+_kustomize_ is a command line tool supporting
-customization of declarative configuration targetted to
+template-free, structured customization of declarative
-k8s-style objects.
+configuration targetted to k8s-style objects.
 
-_Targetted to k8s means_ that kustomize may need some
+_Targetted to k8s means_ that kustomize has some
-limited understanding of API resources, k8s concepts
+understanding of API resources, k8s concepts like
-like names, labels, namespaces, etc. and the semantics
+names, labels, namespaces, etc. and the semantics of
-of resource patching.
+resource patching.
 
 kustomize is an implementation of [DAM].
 
@@ -226,8 +301,8 @@ own [overlays] to do further customization.
 
 ## overlay
 
-An _overlay_ is a [target] that modifies (and thus
+An _overlay_ is a kustomization that modifies (and thus
-depends on) another target.
+depends on) another kustomization.
 
 The [kustomization] in an overlay refers to (via file
 path, URI or other method) some other kustomization,
@@ -245,7 +320,7 @@ _production_ environment variants.
 These variants use the same overall resources, and vary
 in relatively simple ways, e.g. the number of replicas
 in a deployment, the CPU to a particular pod, the data
-source used in a configmap, etc.
+source used in a ConfigMap, etc.
 
 One configures a cluster like this:
 
@@ -260,6 +335,7 @@ One configures a cluster like this:
 Usage of the base is implicit - the overlay's
 kustomization points to the base.
 
+See also [root].
 
 ## package
 
@@ -318,6 +394,14 @@ A _patchJson6902_ can do almost everything a
 _patchStrategicMerge_ can do, but with a briefer
 syntax.  See this [example][patchExampleJson6902].
 
+## plugin
+
+A chunk of code used by kustomize, but not necessarily
+compiled into kustomize, to generate and/or transform a
+kubernetes resource as part of a kustomization.
+
+Details [here][plugins.md].
+
 ## resource
 
 A _resource_ in the context of a REST-ful API is the
@@ -325,15 +409,19 @@ target object of an HTTP operation like _GET_, _PUT_ or
 _POST_.  k8s offers a REST-ful API surface to interact
 with clients.
 
-A _resource_, in the context of kustomization file,
+A _resource_, in the context of a kustomization, is a
-is a path to a [YAML] or [JSON] file describing
+[root] relative path to a [YAML] or [JSON] file
-a k8s API object, like a Deployment or a
+describing a k8s API object, like a Deployment or a
-ConfigmMap.
+ConfigMap, or it's a path to a kustomization, or a URL
+that resolves to a kustomization.
 
 More generally, a resource can be any correct YAML file
 that [defines an object](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/#required-fields)
 with a _kind_ and a _metadata/name_ field.
 
+## root
+
+See [kustomization root][root].
 
 ## sub-target / sub-application / sub-package
 
@@ -348,14 +436,13 @@ The _target_ is the argument to `kustomize build`, e.g.:
 >  kustomize build $target
 > ```
 
-`$target` must be a path or a url to a directory that
+`$target` must be a path or a url to a [kustomization].
-immediately contains a [kustomization].
 
 The target contains, or refers to, all the information
 needed to create customized resources to send to the
 [apply] operation.
 
-A target is a [base] or an [overlay].
+A target can be a [base] or an [overlay].
 
 ## variant
 

docs/plugins.md

@@ -0,0 +1,250 @@
+# kustomize plugins
+
+Kustomize offers a plugin framework for people to
+write their own resource generators (e.g. a helm
+chart processor, a generator that automatically
+attaches a Service and Ingress object to each
+Deployment) and their own resource transformers
+(e.g. a transformer that does some highly
+customized processing of the container command
+line).
+
+## Specification in `kustomization.yaml`
+
+A kustomization file has two new fields in v2.1:
+_generators_ and _transformers_.
+
+Each accepts a list of strings as its arguments:
+
+> ```
+> generators:
+> - relative/path/to/some/file.yaml
+> - relative/path/to/some/kustomization
+> - /absolute/path/to/some/kustomization
+> - https://github.com/org/repo/some/kustomization
+>
+> transformers:
+> - {as above} 
+> ```
+
+This is exactly like the syntax of the `resources` field.
+
+The value of each entry in a `resources`, `generators`
+or `transformers` list must be a relative path to a
+YAML file, or a path or URL to a [kustomization].
+
+[kustomization]: glossary.md#kustomization
+
+In the former case the YAML is read from disk directly,
+and in the latter case a kustomization is performed,
+and its YAML output is merged with the YAML read
+directly from files.  The net result in all three cases
+is an array of YAML objects.
+
+_Each_ object resulting from a `generators` or
+`transformers` field is now further interpreted by
+kustomize as a _plugin configuration_ object.
+
+## Configuration and execution
+
+A kustomization file could have the following lines:
+
+```
+generators:
+- chartInflator.yaml
+```
+
+Given this, the kustomization process would expect to
+find a file called `chartInflator.yaml` in the
+kustomization [root](glossary.md#root).
+
+The file `chartInflator.yaml` could contain:
+
+```
+apiVersion: someteam.example.com/v1
+kind: ChartInflatorExec
+metadata:
+  name: notImportantHere
+chartName: minecraft
+```
+
+The `apiVersion` and `kind` fields of the configuration
+objects are used to _locate_ the plugin.
+
+The rest of the file (actually the entire file) is
+sent to the plugin as configuration - i.e. as the
+plugin's construction arguments.
+
+A kustomization file could include multiple
+instantiations of the same plugin, with different
+arguments (e.g. to inflate two different helm
+charts or two instances of the same chart but with
+different values files).
+
+The value order in the `generators` field doesn't
+matter, because generated objects are just added
+to a sea of objects that kustomize transforms and
+emits.
+
+The specified order of transformers in the
+`transformers` field is, however, respected, as
+transformers aren't expected to be commutative.
+
+Generator plugins are run after processing the
+`resources` field (which _reads_ resources), to
+_create_ additional resources.
+
+The full set of resources is then passed into the
+transformation pipeline, where native (legacy)
+transformations like `namePrefix` and
+`commonLabel` are applied, followed by all the
+transformers run in the order specified.
+
+
+## Placement
+
+[k8s object]: glossary.md#kubernetes-style-object
+
+Given a configuration object (whick looks like any
+other [k8s object]), kustomize will first look for an
+_executable_ file called
+
+```
+$XDG_CONFIG_HOME/kustomize/plugin/${apiVersion}/${kind}
+```
+
+The default value of `XDG_CONFIG_HOME` is `$HOME/.config`.
+
+If this file is not found or is not executable,
+kustomize will look for a file called `${kind}.so`
+in the same directory and attempt to load it as a
+[Go plugin](#go-plugins).
+
+If both checks fails, the plugin load fails the overall
+kustomize build.
+
+A `kustomize build` attempt with plugins that
+omits the flag
+
+TODO: Change flag
+
+> `--enable_alpha_goplugins_accept_panic_risk`
+
+will fail with a warning about plugin use.
+
+Flag use is an opt-in acknowledging the absence of
+plugin provenance.  Its meant to give pause to
+someone who blindly downloads a kustomization from
+the internet and attempts to run it, without
+realizing that it might attempt to run 3rd party
+code.
+
+
+## Writing plugins
+
+### Exec plugins
+
+TODO: Add ptr to example.
+
+A exec plugin is any executable that accepts a
+single argument on it's command line - the name of
+a YAML file containing its configuration (which it
+presumably reads if it needs additional
+configuration).
+
+A generator plugin accepts nothing on `stdin`, but emits
+generated resources to `stdout`.
+
+A transformer plugin accepts resource YAML on `stdin`,
+and emits those resources, possibly transformed, to
+`stdout`.
+
+
+### Go plugins
+
+TODO: Add ptr to example.
+
+[Go plugin]: https://golang.org/pkg/plugin/
+
+A [Go plugin] for kustomize looks like this:
+
+> ```
+> +build plugin
+> 
+> package main
+>
+> import ...
+>
+> // go:generate go run sigs.k8s.io/kustomize/cmd/pluginator
+> type plugin struct{...}
+>
+> var KustomizePlugin plugin
+>
+> func (p *plugin) Config(
+>    ldr ifc.Loader, rf *resmap.Factory,
+>    k ifc.Kunstructured) error {...}
+>
+> func (p *plugin) Generate() (resmap.ResMap, error) {...}
+>
+> func (p *plugin) Transform(m resmap.ResMap) error {...}
+> ```
+
+The use of the identifiers `plugin`,
+`KustomizePlugin` and the three method signatures
+`Configurable`, `Generator`, `Transformer` as
+shown is _required_.
+
+The plugin author should of course change the
+contents of the `plugin` struct, and the three
+method bodies, and the import statements, as
+desired.
+
+Here's a build command, which assumes the plugin
+source code is sitting right next to where the
+shared object (`.so`) files are expected to be:
+
+```
+dir=$XDG_CONFIG_HOME/kustomize/plugin/${apiVersion}
+go build -buildmode plugin -tags=plugin \
+    -o $dir/${kind}.so \
+    $dir/${kind}.go
+```
+
+For the person willing to compile not just a
+plugin but all of kustomze as well, a code
+generator will be provided that will convert a Go
+plugin to statically linked code in your own
+compiled version of kustomize.
+
+#### Caveats
+
+Go plugins allow kustomize extensions that
+
+ * can be tested with the same framework kustomize
+   uses to test its _builtin_ generators and
+   transformers,
+
+ * run without the performance hit of firing up a
+   subprocess and marshalling/unmarshalling data
+   for each plugin run.
+
+Go plugins work as [defined][Go plugin], but
+fall short of what many people think of when they
+hear the word _plugin_.
+
+[ELF]: https://en.wikipedia.org/wiki/Executable_and_Linkable_Format
+
+Go plugin compilation creates an [ELF] formatted
+`.so` file, which by definition has no information
+about the _provenance_ of the file.
+
+One cannot know which version of Go was used,
+which packages were imported (and their version),
+what value of `GOOS` and `GOARCH` were used,
+etc. Skew between the compilation conditions of
+the main program ELF and the plugin ELF will cause
+a failure at load time.
+
+Exec plugins also lack provenance - but they don't
+suffer from the skew problem.
+