kubernetes-sigs/kustomize
1
0
Fork 0
mirror of https://github.com/kubernetes-sigs/kustomize.git synced 2026-06-10 16:42:51 +00:00
Code Issues Packages Projects Releases 1 Wiki Activity

Fully specify KRM Functions Spec and graduate it to v1.

Browse Source 
- Define the OpenAPI schema for ResourceList
- Graduate `ResourceList` to apiVersion `v1`. Many functions have been implemented
  and we are effectively treating the spec as backwards compatible.
- Add `results` field to the spec. This has been implemented in the
  orchestrator and couple of functions libraries for a while, but hasn't been
  yet standardized leading to minor discrepencies. Concurrent to this PR,
  we need to udpate the fn framework (kyaml) to be compatible with
  the standard and update the orchestrator to be backwards compatible.
- Include per-resource annotations (previously a seperate doc)
- Remove support for non-ResourceList wire formats. In practice, this is
  not widely used and complicates the standard. The orchestrators can
  easily covert a List to a ResourceList.
This commit is contained in:
Frank Farzan
2021-05-25 16:09:32 -07:00
parent 5954314b98
commit 3ee1579688
2 changed files with 332 additions and 188 deletions
Download Patch File Download Diff File Expand all files Collapse all files

65
cmd/config/docs/api-conventions/config-io.md
View File

@@ -1,65 +0,0 @@
# Configuration IO API Semantics
Resource Configuration may be read / written from / to sources such as directories,
stdin|out or network. Tools may be composed using pipes such that the tools writing
Resource Configuration may be a different tool from the one that read the configuration.
In order for tools to be composed in this way, while preserving origin information --
such as the original file, index, etc.:
Tools **SHOULD** insert the following annotations when reading from sources,
and **SHOULD** delete the annotations when writing to sinks.
### `config.kubernetes.io/path`
Records the slash-delimited, OS-agnostic, relative file path to a Resource.
This annotation **SHOULD** be set when reading Resources from files.
It **SHOULD** be unset when writing Resources to files.
When writing Resources to a directory, the Resource **SHOULD** be written to the corresponding
path relative to that directory.
Example:
```yaml
metadata:
annotations:
config.kubernetes.io/path: "relative/file/path.yaml"
```
### `config.kubernetes.io/index`
Records the index of a Resource in file. In a multi-object YAML file, Resources are separated
by three dashes (`---`), and the index represents the position of the Resource starting from zero.
This annotation **SHOULD** be set when reading Resources from files.
It **SHOULD** be unset when writing Resources to files.
When writing multiple Resources to the same file, the Resource **SHOULD** be written in the
relative order matching the index.
When this annotation is not specified, it implies a value of `0`.
Example:
```yaml
metadata:
annotations:
config.kubernetes.io/path: "relative/file/path.yaml"
config.kubernetes.io/index: 2
```
This represents the third Resource in the file.
### `config.kubernetes.io/local-config`
`config.kubernetes.io/local-config` declares that the configuration is to local tools
rather than a remote Resource. e.g. The `Kustomization` config in a `kustomization.yaml`
**SHOULD** contain this annotation so that tools know it is not intended to be sent to
the Kubernetes api server.
Example:
```yaml
metadata:
annotations:
config.kubernetes.io/local-config: "true"
```

455
cmd/config/docs/api-conventions/functions-spec.md
View File

@@ -1,13 +1,18 @@
# Configuration Functions Specification
# KRM Configuration Functions Specification
_apiVersion: v1_
## Overview
This document specifies a standard for client-side functions that operate on
Kubernetes declarative configurations. This standard enables creating
small, interoperable, and language-independent executable programs packaged as
containers that can be chained together as part of a configuration management pipeline.
The end result of such a pipeline are fully rendered configurations that can then be
applied to a control plane (e.g. Using ‘kubectl apply’ for Kubernetes control plane).
As such, although this document references Kubernetes Resource Model and API conventions,
it is completely decoupled from Kubernetes API machinery and does not depend on any
Kubernetes declarative configurations referred to as _KRM Functions_. This
standard enables creating small, interoperable, and language-independent
executable programs packaged as containers that can be chained together as part
of a configuration management pipeline. The end result of such a pipeline are
fully rendered configurations that can then be applied to a control plane (e.g.
Using ‘kubectl apply’ for Kubernetes control plane). As such, although this
document references Kubernetes Resource Model and API conventions, it is
completely decoupled from Kubernetes API machinery and does not depend on any
in-cluster components.
This document references terms described in [Kubernetes API Conventions][1].
@@ -18,168 +23,372 @@ interpreted as described in [RFC 2119][2].
## Use Cases
_Configuration functions_ enable shift-left practices (client-side) through:
KRM functions enable shift-left practices (client-side) through:
- Pre-commit / delivery validation and linting of configuration
- e.g. Fail if any containers don't have PodSecurityPolicy or CPU / Memory limits
- Implementation of abstractions as client actuated APIs (e.g. templating)
- e.g. Create a client-side _"CRD"_ for generating configuration checked into git
- Aspect Orient configuration / Injection of cross-cutting configuration
- e.g. T-Shirt size containers by annotating Resources with `small`, `medium`, `large`
and inject the cpu and memory resources into containers accordingly.
- e.g. Inject `init` and `side-car` containers into Resources based off of Resource
Type, annotations, etc.
- e.g. Fail if any containers don't have PodSecurityPolicy or CPU / Memory
limits
- Implementation of abstractions as client actuated APIs
- e.g. Create a client-side _"CRD"_ for generating configuration checked into
git
- Injection of cross-cutting configuration
- e.g. T-Shirt size containers by annotating resources with `small`, `medium`,
`large` and inject the cpu and memory resources into containers accordingly.
- e.g. Inject `init` and `side-car` containers into resources based off of
resource type, annotations, etc.
Performing these on the client rather than the server enables:
- Configuration to be reviewed prior to being sent to the API server
- Configuration to be validated as part of the CI/CD pipeline
- Configuration for Resources to validated holistically rather than individually
per-Resource
- Configuration for resources to validated holistically rather than individually
per-resource
- e.g. ensure the `Service.selector` and `Deployment.spec.template` labels
match.
- e.g. MutatingWebHooks are scoped to a single Resource instance at a time.
- e.g. MutatingWebHooks are scoped to a single resource instance at a time.
- Low-level tweaks to the output of high-level abstractions
- e.g. add an `init container` to a client _"CRD"_ Resource after it was generated.
- e.g. add an `init container` to a client _"CRD"_ resource after it was
generated.
- Composition and layering of multiple functions together
- Compose generation, injection, validation together
## Spec
## Definitions
### Input Type
- **function:** A containerized program conforming to the spec described in this
document.
- **orchestrator:** A program that invokes the function container, passing
arguments and processing its output.
A function MUST accept as input a single [Kubernetes List type][3].
The `items` field in the input will contain a sequence of [Object types][3].
A function MAY not support [Simple types][3] and List types.
## Interface
An example using `v1/ConfigMapList` as input:
The inter-process communication between the orchestrator and a function works as
follows:
1. Orchestrator runs the function container and provides the input on `stdin`.
The input is a Kubernetes object of kind `ResourceList` as described below.
2. Function reads the input from `stdin`, performs computations, and provides
the output as a `ResourceList` to `stdout`. The function MAY also emit
non-structured error message on `stderr`.
3. Orchestrator uses the `stdout`, `stderr`, and the exit code of the function
as it sees fit following to the semantics described below.
### Schema
A function MUST accept input from `stdin` and MUST output to `stdout` a
Kubernetes object of kind `ResourceList` with the following OpenAPI schema:
```yaml
apiVersion: v1
kind: ConfigMapList
items:
- apiVersion: v1
kind: ConfigMap
metadata:
name: config1
data:
p1: v1
p2: v2
- apiVersion: v1
kind: ConfigMap
metadata:
name: config2
swagger: "2.0"
info:
title: KRM Functions Specification (ResourceList)
version: v1
definitions:
ResourceList:
type: object
description: ResourceList is the input/output wire format for KRM functions.
x-kubernetes-group-version-kind:
- group: config.kubernetes.io
kind: ResourceList
version: v1
- group: config.kubernetes.io
kind: ResourceList
version: v1beta1
required:
- items
properties:
apiVersion:
description: apiVersion of ResourceList
type: string
kind:
description: kind of ResourceList i.e. `ResourceList`
type: string
items:
type: array
description: |
[input/output]
Items is a list of Kubernetes objects:
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#types-kinds).
A function will read this field in the input ResourceList and populate
this field in the output ResourceList.
functionConfig:
type: object
description: |
[input]
FunctionConfig is an optional Kubernetes object for passing arguments to a
function invocation.
results:
type: array
description: |
[output]
Results is an optional list that can be used by function to emit results
for observability and debugging purposes.
items:
"$ref": "#/definitions/Result"
Result:
type: object
required:
- message
properties:
message:
type: string
description: Message is a human readable message.
severity:
type: string
enum:
- error
- warning
- info
default: error
description: |
Severity is the severity of a result:
"error": indicates an error result.
"warning": indicates a warning result.
"info": indicates an informational result.
resourceRef:
type: object
description: |
ResourceRef is the metadata for referencing a Kubernetes object
associated with a result.
required:
- apiVersion
- kind
- name
properties:
apiVersion:
description:
APIVersion refers to the `apiVersion` field of the object
manifest.
type: string
kind:
description: Kind refers to the `kind` field of the object.
type: string
namespace:
description:
Namespace refers to the `metadata.namespace` field of the object
manifest.
type: string
name:
description:
Name refers to the `metadata.name` field of the object manifest.
type: string
field:
type: object
description: |
Field is the reference to a field in the object.
If defined, `ResourceRef` must also be provided.
required:
- path
properties:
path:
type: string
description: |
Path is the JSON path of the field
e.g. `spec.template.spec.containers[3].resources.limits.cpu`
currentValue:
description: |
CurrrentValue is the current value of the field.
Can be any value - string, number, boolean, array or object.
proposedValue:
description: |
PropposedValue is the proposed value of the field to fix an issue.
Can be any value - string, number, boolean, array or object.
file:
type: object
description: File references a file containing the resource.
required:
- path
properties:
path:
type: string
description: |
Path is the OS agnostic, slash-delimited, relative path.
e.g. `some-dir/some-file.yaml`.
index:
type: number
default: 0
description: Index of the object in a multi-object YAML file.
tags:
type: object
additionalProperties:
type: string
description: |
Tags is an unstructured key value map stored with a result that may be set
by external tools to store and retrieve arbitrary metadata.
```
An example using `v1/List` as input:
#### Examples
The following is an example input, where the custom resource of kind
`FulfillmentCenter` is provided as `functionConfig`. The function will operate
on one resource of kind `Service`.
```yaml
apiVersion: v1
kind: List
items:
- apiVersion: foo-corp.com/v1
kind: FulfillmentCenter
metadata:
name: staging
address: "100 Main St."
- apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: namespace-reader
rules:
- resources:
- namespaces
apiGroups:
- ""
verbs:
- get
- watch
- list
```
In addition, a function MUST accept as input a List of kind `ResourceList` where the
`functionConfig` field, if present, will contain the invocation-specific configuration passed to the function
by the orchestrator.
Functions MAY consider this field optional so that they can be triggered in an ad-hoc fashion.
An example using `config.kubernetes.io/v1beta1/ResourceList` as input:
```yaml
apiVersion: config.kubernetes.io/v1beta1
apiVersion: config.kubernetes.io/v1
kind: ResourceList
functionConfig:
apiVersion: foo-corp.com/v1
kind: FulfillmentCenter
metadata:
name: staging
metadata:
annotations:
config.kubernetes.io/function: |
container:
image: gcr.io/example/foo:v1.0.0
spec:
address: "100 Main St."
items:
- apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
- apiVersion: v1
kind: Service
metadata:
name: namespace-reader
rules:
- resources:
- namespaces
apiGroups:
- ""
verbs:
- get
- watch
- list
name: wordpress
labels:
app: wordpress
annotations:
config.kubernetes.io/index: "0"
config.kubernetes.io/path: "service.yaml"
spec: # Example comment
type: LoadBalancer
selector:
app: wordpress
tier: frontend
ports:
- protocol: TCP
port: 80
```
Here `FulfillmentCenter` kind with name `staging` is passed as the invocation-specific configuration
to the function.
The following is an example output containing one result representing a
validation error:
### Output Type
A function’s output MUST be the same as the input specification above
-- i.e. `ResourceList` or `List`.
This is necessary to enable chaining two or more functions together in a pipeline.
The serialization format of the output SHOULD match that of its input on each invocation
-- e.g. if the input was a `ResourceList`, the output should also be a `ResourceList`.
```yaml
apiVersion: config.kubernetes.io/v1
kind: ResourceList
items:
- apiVersion: v1
kind: Service
metadata:
name: wordpress
labels:
app: wordpress
annotations:
config.kubernetes.io/index: "0"
config.kubernetes.io/path: "service.yaml"
spec: # Example comment
type: LoadBalancer
selector:
app: wordpress
tier: frontend
ports:
- protocol: TCP
port: 80
results:
- message: "Invalid type. Expected: integer, given: string"
severity: error
resourceRef:
apiVersion: v1
kind: Service
name: wordpress
field:
path: spec.ports.0.port
file:
path: service.yaml
```
### Serialization Format
A function MUST support YAML as a serialization format for the input and output.
A function MUST use utf8 encoding (as YAML is a superset of JSON, JSON will also be supported
by any conforming function).
### Operations
A function MAY Create, Update, or Delete any number of items in the `items` field and output the
resultant list.
A function MAY modify annotations with prefix `config.kubernetes.io`, but must be careful about
doing so since they’re used for orchestration purposes and will likely impact subsequent functions
in the pipeline.
A function SHOULD preserve comments when input serialization format is YAML.
This allows for human authoring of configuration to coexist with changes made by functions.
A function MUST use utf8 encoding (as YAML is a superset of JSON, JSON will also
be supported by any conforming function).
### Containerization
A function MUST be implemented as a container.
A function container MUST be capable of running as a non-root user if it does not require
access to host filesystem or makes network calls.
A function container MUST be capable of running as a non-root user `nobody` if
it does not require access to host filesystem or makes network calls.
### stdin/stdout/stderr and Exit Codes
### stderr
A function MUST accept input from stdin and emit output to stdout.
Any non-structured error messages MUST be emitted to `stderr`. `stdout` is
reserved for `ResourceList` as described above.
Any error messages MUST be emitted to stderr.
### Exit Code
An exit code of zero indicates function execution was successful.
A non-zero exit code indicates a failure.
An exit code of zero indicates function execution was successful. A non-zero
exit code indicates a failure.
[1]: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md
### Operations
A function MAY Create, Update, or Delete any number of items in the `items`
field and output the resultant list in the corresponding `items` field of the
output.
A function MAY modify annotations with prefix `config.kubernetes.io`, but must
be careful about doing so since they’re used for orchestration purposes and will
likely impact subsequent functions in the pipeline. Several annotations and
their semantics are listed below.
A function SHOULD preserve comments when input serialization format is YAML.
This allows for human authoring of configuration to coexist with changes made by
functions.
### Annotations
The orchestrator and function can communicate on a per-resource basis using the
following annotations:
#### `config.kubernetes.io/path`
Records the slash-delimited, OS-agnostic, relative file path to a resource. The
path is relative to a fix location on the filesystem. Different orchestrator
implementations can choose different fixed points.
This annotation only exists in the wire format and is not persisted to the
filesystem. The orchestrator sets this annotation when reading files from the
local filesystem and removes the annotation when writing the output of functions
back to the filesystem.
Example:
```yaml
metadata:
annotations:
config.kubernetes.io/path: "relative/file/path.yaml"
```
#### `config.kubernetes.io/index`
Records the index of a Resource in file. In a multi-object YAML file, resources
are separated by three dashes (`---`), and the index represents the position of
the Resource starting from zero. When this annotation is not specified, it
implies a value of `0`.
Similar to `path` annotation, this annotation is not persisted when writing
configurations files on the file system.
Example:
```yaml
metadata:
annotations:
config.kubernetes.io/path: "relative/file/path.yaml"
config.kubernetes.io/index: 2
```
This represents the third resource in the file.
#### `config.kubernetes.io/local-config`
This annotation declares that the resource is not meant to be applied to a
Kubernetes api server. It is only used for client-side tooling.
Example:
```yaml
metadata:
annotations:
config.kubernetes.io/local-config: "true"
```
[1]:
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md
[2]: https://tools.ietf.org/html/rfc2119
[3]: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#types-kinds
[3]:
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#types-kinds