Compare commits

...

53 Commits

Author SHA1 Message Date
Kubernetes Prow Robot
233f1a3c2a Merge pull request #4344 from natasha41575/ResIdStr
improve gvk and resid strings for error messages
2021-12-22 12:27:03 -08:00
Kubernetes Prow Robot
75de98e2ef Merge pull request #4055 from lstoll/lstoll-unrelated-name-reference
ConfigMapGenerator should not update unrelated ClusterRole rule
2021-12-22 12:15:04 -08:00
Julian
3dbc88bf94 Refactor the PrefixSuffixTransformer into separate prefix- and suffix transformers (#4318)
* Refactor prefix filter into its own filter, decoupled from the prefixsuffix filter

* Refactor prefix transformer into its own transformer, decoupled from the prefixsuffix transformer

* Refactor suffix filter into its own filter, decoupled from the prefixsuffix filter

* Refactor suffix transformer into its own transformer, decoupled from the prefixsuffix transformer

* Add a default nameSuffix field spec in addition to the namePrefix

* Remove the PrefixSuffixTransformer from the list of builtin transformers

* Add a multi-transformer to builtinhelpers.TransformFactories

* Remove the implementation of the prefixsuffixtransformer.PrefixSuffixTransformer

* Resolve style and format related feedback from the pull request

* Add test to test the legacy PrefixSuffixTransformer for BC purposes
2021-12-22 12:01:06 -08:00
Katrina Verey
d701792aa1 Update test demonstrating bug to pass, update comments 2021-12-22 14:53:50 -05:00
Lincoln Stoll
4079056501 ConfigMapGenerator should not update unrelated ClusterRole rule
When using the ConfigMap generator, a lease object entry is updated with the
generated configmap name. This should not happen as it's an unrelated object
type.

As a workaround a unique name can be used for the ConfigMap.

Fails on kustomize version 4.2.0 and kubectl version v1.21.2
2021-12-22 14:47:40 -05:00
natasha41575
6dfc238aa2 improve gvk and resid strings for error messages 2021-12-22 11:39:07 -08:00
Kubernetes Prow Robot
3608f335fd Merge pull request #3924 from monopole/architecture
Add ARCHITECTURE.md
2021-12-22 11:35:04 -08:00
Katrina Verey
56efec5abc Minor updates and clarifications 2021-12-22 14:23:00 -05:00
monopole
2a608bd71c Add ARCHITECTURE.md 2021-12-22 12:07:27 -05:00
Kubernetes Prow Robot
ee4b7847f0 Merge pull request #4221 from devstein/master
docs: Update Plugin Links
2021-12-20 16:37:07 -08:00
Kubernetes Prow Robot
ec38bbeb99 Merge pull request #4342 from natasha41575/moduleSpanDefault
Remove module span check
2021-12-15 11:45:51 -08:00
natasha41575
26999664e6 remove multi-module check 2021-12-15 11:06:38 -08:00
Kubernetes Prow Robot
c0b61b9442 Merge pull request #4332 from natasha41575/OriginCollection
collect origin data only when the option is set
2021-12-13 13:10:26 -08:00
Kubernetes Prow Robot
274a76fe84 Merge pull request #4267 from natasha41575/TransformerAnnotationsProposal
Proposal: Option for transformer annotations
2021-12-13 12:05:58 -08:00
natasha41575
a8f58b4080 collect origin data only when the option is set 2021-12-13 11:38:03 -08:00
natasha41575
542b7c7c4c proposal for transformer annotations option 2021-12-10 12:31:19 -08:00
Kubernetes Prow Robot
9a875add84 Merge pull request #4329 from natasha41575/configMapIssue
[fix] configMapGenerator: extra space in end of line gives "\n" instead of line breaks
2021-12-07 18:07:40 -08:00
natasha41575
a3d547ccd3 fix issue with multiline configmap data with extra space 2021-12-03 15:28:22 -08:00
natasha41575
c4a8a99834 add regression tests for multiline configmap with space 2021-12-03 15:12:21 -08:00
Kubernetes Prow Robot
bc3b249489 Merge pull request #4328 from mengqiy/undeprecate
Remove the deprecation warning for a few RNode helpers
2021-12-03 14:12:36 -08:00
Mengqi Yu
cd2c6a1ad1 Remove the deprecation warning for a few RNode helpers
The Pipe method is not as intuitive as these helpers. These helpers
are convenient and useful. Deprecating them before we have better
alternatives is premature.
2021-12-03 11:47:34 -08:00
Kubernetes Prow Robot
8c6af9440c Merge pull request #4327 from mengqiy/fixanno
use internal.config.kubernetes.io prefix instead of internal.config.k8s.io
2021-12-03 10:02:27 -08:00
Mengqi Yu
6850408f6c update test for cmd/config 2021-12-02 14:00:45 -08:00
Mengqi Yu
ec445049be update tests for api 2021-12-02 14:00:32 -08:00
Mengqi Yu
646915cb86 use internal.config.kubernetes.io prefix instead of internal.config.k8s.io 2021-12-02 14:00:18 -08:00
Kubernetes Prow Robot
fe551be87b Merge pull request #4326 from natasha41575/deprecateStringAndDatePrefixer
deprecate StringPrefixer and DatePrefixer
2021-12-02 10:41:44 -08:00
natasha41575
30280f81af deprecate stringprefixer and dateprefixer 2021-12-02 10:30:56 -08:00
Kubernetes Prow Robot
8cb60f0c5d Merge pull request #4325 from sdowell/kyaml-fn-sort-results
feat: Add Sort method to kyaml fn Results
2021-12-02 10:01:33 -08:00
Sam Dowell
9d29f57789 feat: Add Sort method to kyaml fn Results
Issues: GoogleContainerTools/kpt#2195
2021-12-01 20:57:48 +00:00
Kubernetes Prow Robot
a3e1c99915 Merge pull request #4307 from sdowell/annotations-filter-set-entry
feat: Add SetEntryCallback to annotations filter
2021-11-29 18:20:56 -08:00
Sam Dowell
0fe1236e20 feat: Add SetEntryCallback to annotations filter
Add a configurable callback that is invoked each time an
annotation is applied by the annotations filter. This is useful
for scenarios such as tracking annotations as they are applied.

Issues: GoogleContainerTools/kpt#2448
2021-11-30 01:21:42 +00:00
Kubernetes Prow Robot
25ee506af4 Merge pull request #4300 from yuwenma/refactor-builtin
Move api/builtin to internal
2021-11-22 16:04:16 -08:00
Yuwen Ma
a1c5d79d94 Move api/builtin to internal 2021-11-22 15:54:18 -08:00
Kubernetes Prow Robot
de5210b43a Merge pull request #4299 from yuwenma/nameReference-doc
[Doc] Improve `nameReference`  docs, examples and builtin references.
2021-11-22 13:25:32 -08:00
Yuwen Ma
d9c4c749e2 improve docs with examples for nameReference 2021-11-22 13:10:13 -08:00
Natasha Sarkar
01420768c8 Merge pull request #4241 from Goodwine/kyaml-result-provider
Handle "Result" errors as non-fatal errors in kyaml FilterFuncs
2021-11-22 11:07:18 -08:00
Natasha Sarkar
d11342489a Merge pull request #4297 from mengqiy/anno
kyaml/fn/framework ensures the annotation output format matches the input
2021-11-19 12:37:01 -08:00
Mengqi Yu
bd7bad19a1 add to BuildAnnotations 2021-11-19 12:13:12 -08:00
Mengqi Yu
dfc627068b update tests for cmd/config module 2021-11-19 07:30:31 -08:00
Mengqi Yu
166c2e766b update tests for api module 2021-11-19 07:30:31 -08:00
Mengqi Yu
2ee2d3e389 use custom id as key of the mapping and make kio.Pipeline behave the same as framework.Execute 2021-11-19 07:30:31 -08:00
Mengqi Yu
81edfb7ee8 default result.severity to info when formatting string 2021-11-19 07:30:31 -08:00
Mengqi Yu
4e7aebc20c address comments 2021-11-18 14:58:04 -08:00
Mengqi Yu
5caed5b90a kyaml/fn/framework ensures the annotation output format matches the input
If the input only contains legacy format anntations (path, index, id), the
output will be the same.
2021-11-17 14:09:19 -08:00
Goodwine
4cde50ab14 fix typo
Co-authored-by: Katrina Verey <kn.verey@gmail.com>
2021-11-11 16:33:17 -08:00
Kubernetes Prow Robot
2f115223cc Merge pull request #4285 from KnVerey/updateProwExamplesTarget
Test examples against latest release
2021-11-11 16:12:28 -08:00
Katrina Verey
92c505a211 Test examples against latest release 2021-11-11 16:00:51 -08:00
Kubernetes Prow Robot
e9ea7657ee Merge pull request #4284 from KnVerey/unpinEverything
Back to development mode; unpin the modules
2021-11-11 15:59:15 -08:00
Katrina Verey
4bcc57de74 Back to development mode; unpin the modules 2021-11-11 15:49:01 -08:00
Kubernetes Prow Robot
b2d65ddc98 Merge pull request #4283 from KnVerey/pinToApi
Pin to api v0.10.1
2021-11-11 15:27:14 -08:00
Carlos Ortiz García
894ffec36a Handle kyaml Filter errors type Result as non-breaking errors and store in ResourceList
- Result can only count as error when passed as pointer, this makes easy use of "errors.As"
- Existing Filter() implementations that return Result from the `framework` package won't return an error anymore but modify the ResourceList
2021-11-11 15:26:07 -08:00
Katrina Verey
3db4a94281 Pin to api v0.10.1 2021-11-11 15:18:05 -08:00
Devin Stein
3479b6691e docs: Update plugin links 2021-10-04 12:58:16 -07:00
110 changed files with 3674 additions and 1286 deletions

299
ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,299 @@
# Architecture
* _Updated: December 2021_
This document describes the repository organization and the kustomize
build process. It's meant to lower the barrier to learning and
contributing to the code base.
If not kept up to date, it will just be a historical snapshot.
## Repository layout
[human-edited docs]: https://github.com/kubernetes-sigs/cli-experimental/tree/master/site
[generated docs]: https://github.com/kubernetes-sigs/cli-experimental/tree/master/docs
[rendered docs]: https://kubectl.docs.kubernetes.io
[openapi]: https://kubernetes.io/blog/2016/12/kubernetes-supports-openapi
[`api` module]: https://github.com/kubernetes-sigs/kustomize/blob/master/api/go.mod
[`api`]: #the-api-module
[`cmd/config` module]: https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/go.mod
[`cmd/config`]: #the-cmdconfig-module
[`kustomize` module]: https://github.com/kubernetes-sigs/kustomize/blob/master/kustomize/go.mod
[`kustomize`]: #the-kustomize-module
[`kyaml` module]: https://github.com/kubernetes-sigs/kustomize/blob/master/kyaml/go.mod
[`kyaml`]: #the-kyaml-module
[`kyaml/kio.Filter`]: https://github.com/Kubernetes-sigs/kustomize/blob/master/kyaml/kio/kio.go
[`go-yaml`]: https://github.com/go-yaml/yaml/tree/v3
[3922]: https://github.com/kubernetes-sigs/kustomize/issues/3922
| directory | purpose |
| ---------: | :---------- |
| `api` | The [`api`] module, holding high level kustomize code, suitable for import by other programs. |
| `cmd` | Various Go programs aiding repo management. See also `hack`. As an outlier, includes the special [`cmd/config`] module. |
| `docs` | Old home of documentation; contains pointers to new homes: [human-edited docs], [generated docs] and [rendered docs]. |
| `examples` | Full kustomization examples that run as pre-merge tests. |
| `functions` | Examples of plugins in KRM function form. TODO([3922]): Move under `plugin`. |
| `hack` | Various shell scripts to help with code management. |
| `kustomize` | The [`kustomize`] module holds the `main.go` for kustomize. |
| `kyaml` | The [`kyaml`] module, holding Kubernetes-specific YAML editing packages used by the [`api`] module. Wraps [`go-yaml`] v3.|
| `plugin` | Examples of Kustomize plugins. |
| `releasing` | Instructions for releasing the various modules. |
| `site` | Old generated documentation, kept to provide redirection links to the new docs. |
## Modules
[semantically versioned]: https://semver.org
[Go modules]: https://github.com/golang/go/wiki/Modules
The [Go modules] in the kustomize repository are [semantically versioned].
### `kustomize`
> _Depends on [`api`], [`cmd/config`], [`kyaml`]_
The [`kustomize` module] contains the `main.go` for `kustomize`, buildable with
```
(cd kustomize; go install .)
```
[appears in kubectl]: https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/kubectl/pkg/cmd/kustomize/kustomize.go
Below this are packages containing
[cobra](http://github.com/spf13/cobra) commands implementing `build`,
`edit`, `fix`, etc., packages linked together by `main.go`.
These command packages are intentionally public, semantically
versioned, and can be used in other programs. Specifically, the
`kustomize build` command [appears in kubectl] as `kubectl kustomize`.
The code in the `build` package is dominated by flag validation,
with minimal business logic. The critical lines are something
like
```
# Make a kustomizer.
k := krusty.MakeKustomizer(
HonorKustomizeFlags(krusty.MakeDefaultOptions()),
)
# Run the kustomizer, sending location of kustomization.yaml
m := k.Run(fSys, "/path/to/dir")
# Write the result as YAML.
writer.Write(m.AsYaml())
```
The `krusty` package is in the [`api`] module.
### `api`
> _Depends on [`kyaml`] and code generated from builtin plugin modules_
The [`api` module] is used by CLI programs like `kustomize` and `kubectl`
to read and honor `kustomization.yaml` files and all that implies.
The main public packages in the [`api` module] are
| package | |
| --------: | :---------- |
| `filters` | Implementations of [`kyaml/kio.Filter`] used by kustomize to transform Kubernetes objects. |
| `konfig` | Configuration methods and constants in the kustomize API. |
| `krusty` | Primary API entry point. Holds the kustomizer and hundreds of tests for it. |
| `loader` | Loads kustomization files and the files they refer to, enforcing security rules. |
| `resmap` | The primary internal data structure over which the kustomizer and filters work. |
| `types` | The `Kustomization` object and ancillary structs. |
### `cmd/config`
> _Depends on [`kyaml`]_
This module contains cobra commands and kyaml-based functionality to
provide unix-like file manipulation commands to kustomize like `grep`
and `tree`. These commands may be included in any program that
manipulates k8s YAML (e.g. kustomize).
### `kyaml`
> _Has no in-repo dependence_
The [`kyaml` module] is a kubernetes-focussed enhancement of [go-yaml].
The YAML manipulation performed by a kustomize is based on these libraries.
These libraries evolve independently of kustomize, and other programs depend on them.
The key public packages in the [`kyaml` module] include
| package | |
| --------: | :---------- |
| `errors` | Wrapper for the go-errors/errors lib |
| `filesys` | A kustomize-specific file system abstraction, to ease writing tests |
| `fn/framework` | An SDK for writing KRM Functions in Go |
| `fn/runtime` | Implements the runtime for KRM Function extensions |
| `kio` | Libraries for reading and writing collections of Kubernetes resources as RNodes |
| `openapi` | Loads and accesses openapi schemas for schema-aware resource manipultaion |
| `resid` | Representations to aid in unique identification of Kubernetes resources |
| `yaml` | A Kubernetes-focused wrapper of [go-yaml], notably including the RNode object |
-------
## How _kustomize build_ works
The command `kustomize build` accepts a single string argument,
which must resolve to a directory, possibly in a git repository,
called the _kustomization root_.
This directory must contain a file called `kustomization.yaml`, with
YAML that marshals into a single instance of a `Kustomization` object.
For the remainder of this document, the word _kustomization_ refers to
either of these things.
This kustomization is the access point to a directed, acyclic graph of
Kubernetes objects, including other kustomizations, to include in a
build.
Execution of `build` starts and ends in the [`api`] module,
frequently dipping into the [`kyaml`] module for lower level
YAML manipulation.
### The `build` flow
- Validate command lines arguments and flags.
- Make a `Kustomizer` as a function of those arguments.
- Call `Run` on the kustomizer, passing it the path to the
kustomization.
`Run` returns an instance of `ResMap`, the `api` package's
representation of a set of kubernetes `Resource` objects.
This structure offers resource lookup methods (map behavior),
but also retains the resources in the order they were
specified in kustomization files (list behavior).
Post-run, the objects are fully hydrated, per the
instructions in the kustomization.
- Marshal the objects as YAML to a file or `stdout`.
### The `Run` function
- Create various objects
- A `ResMap` factory.
Makes `ResMaps` from byte streams, other `ResMaps`, etc.
- A file `loader.Loader`.
It's fed an appropriate set of restrictions, and the path to the kustomization.
- A plugin loader.
It finds plugins (transformers, generators or validators)
and prepares them for running.
- A `KustTarget` encapsulating all of the above.
A KustTarget contains one `Kustomization` and represents
everything that kustomization can reach. This will include
other `KustTarget` instances, each having a smaller purview than
the one referencing it.
- Call `KustTarget.Load` to load its kustomization.
This step deals with deprecations and field changes.
- Load [openapi] data specified by the kustomization.
This is needed to recognize k8s kinds and their special
properties, e.g. which kinds are cluster-scoped, which kinds
refer to others, etc.
- Call `KustTarget.makeCustomizedResmap` to create the `ResMap` result.
This visits everything referenced by the kustomization,
performing all generation, transformation and validation.
- Finish the `Run` with
- Optional reordering of objects in `ResMap`, overriding the
FIFO rule.
- Optional addition of _kustomize build annotations_ to the
resources. E.g. from which repo and file the resource was
read, the fact that kustomize touched the resource, etc.
These kustomize-specific annotations are intended for
server-side data analytics, file structure traceability and
reconstruction, etc.
### The `makeCustomizedResmap` function
This function starts the process of object transformation,
as well as accumulation of recursively referenced data.
- Call `ra := KustTarget.AccumulateTarget`.
The result, `ra`, is a resource accumulator that contains
everything referred to by the current kustomization, now fully
hydrated.
- Uniquify names of generated objects by appending content hashes.
This cannot be done until the objects are complete.
- Fix all name references (given that names may have changed).
E.g. if a ConfigMaps was given a generated name, all objects that
refer to that ConfigMap must be given its name.
- Resolve vars, replacing them with whatever they refer to (a legacy feature).
### The `AccumulateTarget` function
- Call `AccumulateResources` over the `resources` field (this can recurse).
- Call `AccumulateComponents` over the `components` field (this can recurse),
- Load legacy (pre-plugin) global kustomize configuration,
- Load legacy (pre-openapi) _Custom Resource Definition_ data.
- In the context of the data loaded above, run the kustomization's
- generators,
- transformers,
- and validators.
- Accumulate `vars` (make note of them for later replacement).
### `AccumulateResources` and component accumulation
- If the path is a file:
- Accumulate the objects in the file (treating them
as opaque kubernetes objects).
- If the path is a directory:
- Create a new `KustTarget` referring to that directory's kustomization.
- Call `subRa := KustTarget.AccumulateTarget`.
- Call `ra.MergeAccumulator(subRa)`
This completes a recursion.
- If the path is a git URL:
- Clone the repository to a temporary directory.
- Process the path optionally specified in the URL
as a path in the clone.
- If no path specified, work from the repository root.
That's as deep as this discussion will go.
The deeper this document goes into the details, the faster
it will get out of date.

View File

@@ -12,7 +12,7 @@ MYGOBIN = $(shell go env GOPATH)/bin
endif
export PATH := $(MYGOBIN):$(PATH)
MODULES := '"cmd/config" "api/" "kustomize/" "kyaml/"'
LATEST_V4_RELEASE=v4.4.0
LATEST_V4_RELEASE=v4.4.1
# Provide defaults for REPO_OWNER and REPO_NAME if not present.
# Typically these values would be provided by Prow.
@@ -40,7 +40,6 @@ verify-kustomize: \
prow-presubmit-check: \
install-tools \
lint-kustomize \
test-multi-module \
test-unit-kustomize-all \
test-unit-cmd-all \
test-go-mod \
@@ -83,11 +82,6 @@ $(MYGOBIN)/pluginator:
cd cmd/pluginator; \
go install .
# Build from local source.
$(MYGOBIN)/prchecker:
cd cmd/prchecker; \
go install .
# Build from local source.
$(MYGOBIN)/kustomize: build-kustomize-api
cd kustomize; \
@@ -102,7 +96,6 @@ install-tools: \
$(MYGOBIN)/k8scopy \
$(MYGOBIN)/mdrip \
$(MYGOBIN)/pluginator \
$(MYGOBIN)/prchecker \
$(MYGOBIN)/stringer
### Begin kustomize plugin rules.
@@ -127,7 +120,7 @@ install-tools: \
# module (it's linked into the api).
# Where all generated builtin plugin code should go.
pGen=api/builtins
pGen=api/internal/builtins
# Where the builtin Go plugin modules live.
pSrc=plugin/builtin
@@ -143,7 +136,8 @@ _builtinplugins = \
PatchJson6902Transformer.go \
PatchStrategicMergeTransformer.go \
PatchTransformer.go \
PrefixSuffixTransformer.go \
PrefixTransformer.go \
SuffixTransformer.go \
ReplacementTransformer.go \
ReplicaCountTransformer.go \
SecretGenerator.go \
@@ -171,7 +165,8 @@ $(pGen)/NamespaceTransformer.go: $(pSrc)/namespacetransformer/NamespaceTransform
$(pGen)/PatchJson6902Transformer.go: $(pSrc)/patchjson6902transformer/PatchJson6902Transformer.go
$(pGen)/PatchStrategicMergeTransformer.go: $(pSrc)/patchstrategicmergetransformer/PatchStrategicMergeTransformer.go
$(pGen)/PatchTransformer.go: $(pSrc)/patchtransformer/PatchTransformer.go
$(pGen)/PrefixSuffixTransformer.go: $(pSrc)/prefixsuffixtransformer/PrefixSuffixTransformer.go
$(pGen)/PrefixTransformer.go: $(pSrc)/prefixtransformer/PrefixTransformer.go
$(pGen)/SuffixTransformer.go: $(pSrc)/suffixtransformer/SuffixTransformer.go
$(pGen)/ReplacementTransformer.go: $(pSrc)/replacementtransformer/ReplacementTransformer.go
$(pGen)/ReplicaCountTransformer.go: $(pSrc)/replicacounttransformer/ReplicaCountTransformer.go
$(pGen)/SecretGenerator.go: $(pSrc)/secretgenerator/SecretGenerator.go
@@ -252,19 +247,6 @@ test-unit-cmd-all:
test-go-mod:
./hack/check-go-mod.sh
# Environment variables are defined at
# https://github.com/kubernetes/test-infra/blob/master/prow/jobs.md#job-environment-variables
.PHONY: test-multi-module
test-multi-module: $(MYGOBIN)/prchecker
( \
export MYGOBIN=$(MYGOBIN); \
export REPO_OWNER=$(REPO_OWNER); \
export REPO_NAME=$(REPO_NAME); \
export PULL_NUMBER=$(PULL_NUMBER); \
export MODULES=$(MODULES); \
./hack/check-multi-module.sh; \
)
.PHONY:
test-examples-e2e-kustomize: $(MYGOBIN)/mdrip $(MYGOBIN)/kind
( \
@@ -358,7 +340,6 @@ clean: clean-kustomize-external-go-plugin
rm -f $(MYGOBIN)/golangci-lint-kustomize
rm -f $(MYGOBIN)/kustomize
rm -f $(MYGOBIN)/mdrip
rm -f $(MYGOBIN)/prchecker
rm -f $(MYGOBIN)/stringer
# Handle pluginator manually.

51
api/builtins/builtins.go Normal file
View File

@@ -0,0 +1,51 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Deprecated: Package api/builtins will not be available in API v1.
package builtins
import (
internal "sigs.k8s.io/kustomize/api/internal/builtins"
)
type (
AnnotationsTransformerPlugin = internal.AnnotationsTransformerPlugin
ConfigMapGeneratorPlugin = internal.ConfigMapGeneratorPlugin
HashTransformerPlugin = internal.HashTransformerPlugin
HelmChartInflationGeneratorPlugin = internal.HelmChartInflationGeneratorPlugin
IAMPolicyGeneratorPlugin = internal.IAMPolicyGeneratorPlugin
ImageTagTransformerPlugin = internal.ImageTagTransformerPlugin
LabelTransformerPlugin = internal.LabelTransformerPlugin
LegacyOrderTransformerPlugin = internal.LegacyOrderTransformerPlugin
NamespaceTransformerPlugin = internal.NamespaceTransformerPlugin
PatchJson6902TransformerPlugin = internal.PatchJson6902TransformerPlugin
PatchStrategicMergeTransformerPlugin = internal.PatchStrategicMergeTransformerPlugin
PatchTransformerPlugin = internal.PatchTransformerPlugin
PrefixTransformerPlugin = internal.PrefixTransformerPlugin
SuffixTransformerPlugin = internal.SuffixTransformerPlugin
ReplacementTransformerPlugin = internal.ReplacementTransformerPlugin
ReplicaCountTransformerPlugin = internal.ReplicaCountTransformerPlugin
SecretGeneratorPlugin = internal.SecretGeneratorPlugin
ValueAddTransformerPlugin = internal.ValueAddTransformerPlugin
)
var (
NewAnnotationsTransformerPlugin = internal.NewAnnotationsTransformerPlugin
NewConfigMapGeneratorPlugin = internal.NewConfigMapGeneratorPlugin
NewHashTransformerPlugin = internal.NewHashTransformerPlugin
NewHelmChartInflationGeneratorPlugin = internal.NewHelmChartInflationGeneratorPlugin
NewIAMPolicyGeneratorPlugin = internal.NewIAMPolicyGeneratorPlugin
NewImageTagTransformerPlugin = internal.NewImageTagTransformerPlugin
NewLabelTransformerPlugin = internal.NewLabelTransformerPlugin
NewLegacyOrderTransformerPlugin = internal.NewLegacyOrderTransformerPlugin
NewNamespaceTransformerPlugin = internal.NewNamespaceTransformerPlugin
NewPatchJson6902TransformerPlugin = internal.NewPatchJson6902TransformerPlugin
NewPatchStrategicMergeTransformerPlugin = internal.NewPatchStrategicMergeTransformerPlugin
NewPatchTransformerPlugin = internal.NewPatchTransformerPlugin
NewPrefixTransformerPlugin = internal.NewPrefixTransformerPlugin
NewSuffixTransformerPlugin = internal.NewSuffixTransformerPlugin
NewReplacementTransformerPlugin = internal.NewReplacementTransformerPlugin
NewReplicaCountTransformerPlugin = internal.NewReplicaCountTransformerPlugin
NewSecretGeneratorPlugin = internal.NewSecretGeneratorPlugin
NewValueAddTransformerPlugin = internal.NewValueAddTransformerPlugin
)

View File

@@ -19,10 +19,25 @@ type Filter struct {
// FsSlice contains the FieldSpecs to locate the namespace field
FsSlice types.FsSlice
// SetEntryCallback is invoked each time an annotation is applied
// Example use cases:
// - Tracking all paths where annotations have been applied
SetEntryCallback func(key, value, tag string, node *yaml.RNode)
}
var _ kio.Filter = Filter{}
func (f Filter) setEntry(key, value, tag string) filtersutil.SetFn {
baseSetEntryFunc := filtersutil.SetEntry(key, value, tag)
return func(node *yaml.RNode) error {
if f.SetEntryCallback != nil {
f.SetEntryCallback(key, value, tag, node)
}
return baseSetEntryFunc(node)
}
}
func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
keys := yaml.SortedMapKeys(f.Annotations)
_, err := kio.FilterAll(yaml.FilterFunc(
@@ -30,7 +45,7 @@ func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
for _, k := range keys {
if err := node.PipeE(fsslice.Filter{
FsSlice: f.FsSlice,
SetValue: filtersutil.SetEntry(
SetValue: f.setEntry(
k, f.Annotations[k], yaml.NodeTagString),
CreateKind: yaml.MappingNode, // Annotations are MappingNodes.
CreateTag: yaml.NodeTagMap,

View File

@@ -11,16 +11,36 @@ import (
"sigs.k8s.io/kustomize/api/internal/plugins/builtinconfig"
filtertest_test "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
var annosFs = builtinconfig.MakeDefaultConfig().CommonAnnotations
type setEntryArg struct {
Key string
Value string
Tag string
NodePath []string
}
var setEntryArgs []setEntryArg
func setEntryCallbackStub(key, value, tag string, node *yaml.RNode) {
setEntryArgs = append(setEntryArgs, setEntryArg{
Key: key,
Value: value,
Tag: tag,
NodePath: node.FieldPath(),
})
}
func TestAnnotations_Filter(t *testing.T) {
testCases := map[string]struct {
input string
expectedOutput string
filter Filter
fsslice types.FsSlice
input string
expectedOutput string
filter Filter
fsslice types.FsSlice
expectedSetEntryArgs []setEntryArg
}{
"add": {
input: `
@@ -210,9 +230,74 @@ metadata:
"b": "b1",
}},
},
// test usage of SetEntryCallback
"set_entry_callback": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
`,
expectedOutput: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
annotations:
a: a1
b: b1
spec:
template:
metadata:
annotations:
a: a1
b: b1
`,
filter: Filter{
Annotations: annoMap{
"a": "a1",
"b": "b1",
},
SetEntryCallback: setEntryCallbackStub,
},
fsslice: []types.FieldSpec{
{
Path: "spec/template/metadata/annotations",
CreateIfNotPresent: true,
},
},
expectedSetEntryArgs: []setEntryArg{
{
Key: "a",
Value: "a1",
Tag: "!!str",
NodePath: []string{"metadata", "annotations"},
},
{
Key: "a",
Value: "a1",
Tag: "!!str",
NodePath: []string{"spec", "template", "metadata", "annotations"},
},
{
Key: "b",
Value: "b1",
Tag: "!!str",
NodePath: []string{"metadata", "annotations"},
},
{
Key: "b",
Value: "b1",
Tag: "!!str",
NodePath: []string{"spec", "template", "metadata", "annotations"},
},
},
},
}
for tn, tc := range testCases {
setEntryArgs = nil
t.Run(tn, func(t *testing.T) {
filter := tc.filter
filter.FsSlice = append(annosFs, tc.fsslice...)
@@ -221,6 +306,9 @@ metadata:
strings.TrimSpace(filtertest_test.RunFilter(t, tc.input, filter))) {
t.FailNow()
}
if !assert.Equal(t, tc.expectedSetEntryArgs, setEntryArgs) {
t.FailNow()
}
})
}
}

View File

@@ -63,6 +63,9 @@ xxx:
apiVersion: foo/v1
kind: Bar
xxx:
metadata:
annotations:
internal.config.kubernetes.io/annotations-migration-resource-id: '0'
: cannot set or create an empty field name`,
filter: fieldspec.Filter{
SetValue: filtersutil.SetScalar("e"),
@@ -220,6 +223,9 @@ a:
kind: Bar
a:
b: a
metadata:
annotations:
internal.config.kubernetes.io/annotations-migration-resource-id: '0'
: expected sequence or mapping node`,
filter: fieldspec.Filter{
SetValue: filtersutil.SetScalar("e"),

View File

@@ -0,0 +1,6 @@
// Copyright 2020 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package prefix contains a kio.Filter implementation of the kustomize
// PrefixTransformer.
package prefix

View File

@@ -1,14 +1,14 @@
// Copyright 2020 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package prefixsuffix_test
package prefix_test
import (
"bytes"
"log"
"os"
"sigs.k8s.io/kustomize/api/filters/prefixsuffix"
"sigs.k8s.io/kustomize/api/filters/prefix"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
)
@@ -26,7 +26,7 @@ kind: Bar
metadata:
name: instance
`)}},
Filters: []kio.Filter{prefixsuffix.Filter{
Filters: []kio.Filter{prefix.Filter{
Prefix: "baz-", FieldSpec: types.FieldSpec{Path: "metadata/name"}}},
Outputs: []kio.Writer{kio.ByteWriter{Writer: os.Stdout}},
}.Execute()

View File

@@ -1,7 +1,7 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package prefixsuffix
package prefix
import (
"fmt"
@@ -13,10 +13,9 @@ import (
"sigs.k8s.io/kustomize/kyaml/yaml"
)
// Filter applies resource name prefix's and suffix's using the fieldSpecs
// Filter applies resource name prefix's using the fieldSpecs
type Filter struct {
Prefix string `json:"prefix,omitempty" yaml:"prefix,omitempty"`
Suffix string `json:"suffix,omitempty" yaml:"suffix,omitempty"`
FieldSpec types.FieldSpec `json:"fieldSpec,omitempty" yaml:"fieldSpec,omitempty"`
}
@@ -39,5 +38,5 @@ func (f Filter) run(node *yaml.RNode) (*yaml.RNode, error) {
func (f Filter) evaluateField(node *yaml.RNode) error {
return filtersutil.SetScalar(fmt.Sprintf(
"%s%s%s", f.Prefix, node.YNode().Value, f.Suffix))(node)
"%s%s", f.Prefix, node.YNode().Value))(node)
}

View File

@@ -1,14 +1,14 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package prefixsuffix_test
package prefix_test
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"sigs.k8s.io/kustomize/api/filters/prefixsuffix"
"sigs.k8s.io/kustomize/api/filters/prefix"
filtertest_test "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
)
@@ -37,62 +37,10 @@ kind: Bar
metadata:
name: foo-instance
`,
filter: prefixsuffix.Filter{Prefix: "foo-"},
fs: types.FieldSpec{Path: "metadata/name"},
},
"suffix": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance-foo
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance-foo
`,
filter: prefixsuffix.Filter{Suffix: "-foo"},
fs: types.FieldSpec{Path: "metadata/name"},
},
"prefix-suffix": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: bar-instance-foo
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: bar-instance-foo
`,
filter: prefixsuffix.Filter{Prefix: "bar-", Suffix: "-foo"},
fs: types.FieldSpec{Path: "metadata/name"},
filter: prefix.Filter{
Prefix: "foo-",
FieldSpec: types.FieldSpec{Path: "metadata/name"},
},
},
"data-fieldspecs": {
@@ -130,23 +78,23 @@ a:
b:
c: foo-d
`,
filter: prefixsuffix.Filter{Prefix: "foo-"},
fs: types.FieldSpec{Path: "a/b/c"},
filter: prefix.Filter{
Prefix: "foo-",
FieldSpec: types.FieldSpec{Path: "a/b/c"},
},
},
}
type TestCase struct {
input string
expected string
filter prefixsuffix.Filter
fs types.FieldSpec
filter prefix.Filter
}
func TestFilter(t *testing.T) {
for name := range tests {
test := tests[name]
t.Run(name, func(t *testing.T) {
test.filter.FieldSpec = test.fs
if !assert.Equal(t,
strings.TrimSpace(test.expected),
strings.TrimSpace(

View File

@@ -1,6 +0,0 @@
// Copyright 2020 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package prefixsuffix contains a kio.Filter implementation of the kustomize
// PrefixSuffixTransformer.
package prefixsuffix

View File

@@ -251,6 +251,7 @@ metadata:
annotations:
config.kubernetes.io/index: '0'
internal.config.kubernetes.io/index: '0'
internal.config.kubernetes.io/annotations-migration-resource-id: '0'
data:
slice:
- false
@@ -278,6 +279,7 @@ metadata:
annotations:
config.kubernetes.io/index: '0'
internal.config.kubernetes.io/index: '0'
internal.config.kubernetes.io/annotations-migration-resource-id: '0'
data:
1: str
: invalid map key: value='1', tag='` + yaml.NodeTagInt + `'`,

View File

@@ -172,7 +172,7 @@ func getReplacement(nodes []*yaml.RNode, r *types.Replacement) (*yaml.RNode, err
return nil, err
}
if rn.IsNilOrEmpty() {
return nil, fmt.Errorf("fieldPath `%s` is missing for replacement source %s", r.Source.FieldPath, r.Source)
return nil, fmt.Errorf("fieldPath `%s` is missing for replacement source %s", r.Source.FieldPath, r.Source.ResId)
}
return getRefinedValue(r.Source.Options, rn)

View File

@@ -269,7 +269,7 @@ spec:
- select:
kind: Deployment
`,
expectedErr: "multiple matches for selector ~G_~V_Deployment|~X|~N",
expectedErr: "multiple matches for selector Deployment.[noVer].[noGrp]/[noName].[noNs]",
},
"replacement has no source": {
input: `apiVersion: v1
@@ -1586,7 +1586,7 @@ data:
options:
create: true
`,
expectedErr: "fieldPath `data.httpPort` is missing for replacement source ~G_~V_ConfigMap|~X|ports-from:data.httpPort",
expectedErr: "fieldPath `data.httpPort` is missing for replacement source ConfigMap.[noVer].[noGrp]/ports-from.[noNs]",
},
"annotationSelector": {
input: `apiVersion: v1

View File

@@ -0,0 +1,6 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package suffix contains a kio.Filter implementation of the kustomize
// SuffixTransformer.
package suffix

View File

@@ -0,0 +1,47 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package suffix_test
import (
"bytes"
"log"
"os"
"sigs.k8s.io/kustomize/api/filters/suffix"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
)
func ExampleFilter() {
err := kio.Pipeline{
Inputs: []kio.Reader{&kio.ByteReader{Reader: bytes.NewBufferString(`
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`)}},
Filters: []kio.Filter{suffix.Filter{
Suffix: "-baz", FieldSpec: types.FieldSpec{Path: "metadata/name"}}},
Outputs: []kio.Writer{kio.ByteWriter{Writer: os.Stdout}},
}.Execute()
if err != nil {
log.Fatal(err)
}
// Output:
// apiVersion: example.com/v1
// kind: Foo
// metadata:
// name: instance-baz
// ---
// apiVersion: example.com/v1
// kind: Bar
// metadata:
// name: instance-baz
}

View File

@@ -0,0 +1,42 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package suffix
import (
"fmt"
"sigs.k8s.io/kustomize/api/filters/fieldspec"
"sigs.k8s.io/kustomize/api/filters/filtersutil"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
// Filter applies resource name suffix's using the fieldSpecs
type Filter struct {
Suffix string `json:"suffix,omitempty" yaml:"suffix,omitempty"`
FieldSpec types.FieldSpec `json:"fieldSpec,omitempty" yaml:"fieldSpec,omitempty"`
}
var _ kio.Filter = Filter{}
func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
return kio.FilterAll(yaml.FilterFunc(f.run)).Filter(nodes)
}
func (f Filter) run(node *yaml.RNode) (*yaml.RNode, error) {
err := node.PipeE(fieldspec.Filter{
FieldSpec: f.FieldSpec,
SetValue: f.evaluateField,
CreateKind: yaml.ScalarNode, // Name is a ScalarNode
CreateTag: yaml.NodeTagString,
})
return node, err
}
func (f Filter) evaluateField(node *yaml.RNode) error {
return filtersutil.SetScalar(fmt.Sprintf(
"%s%s", node.YNode().Value, f.Suffix))(node)
}

View File

@@ -0,0 +1,106 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package suffix_test
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"sigs.k8s.io/kustomize/api/filters/suffix"
filtertest_test "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
)
var tests = map[string]TestCase{
"suffix": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance-foo
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance-foo
`,
filter: suffix.Filter{
Suffix: "-foo",
FieldSpec: types.FieldSpec{Path: "metadata/name"},
},
},
"data-fieldspecs": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
a:
b:
c: d
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
a:
b:
c: d
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
a:
b:
c: d-foo
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
a:
b:
c: d-foo
`,
filter: suffix.Filter{
Suffix: "-foo",
FieldSpec: types.FieldSpec{Path: "a/b/c"},
},
},
}
type TestCase struct {
input string
expected string
filter suffix.Filter
}
func TestFilter(t *testing.T) {
for name := range tests {
test := tests[name]
t.Run(name, func(t *testing.T) {
if !assert.Equal(t,
strings.TrimSpace(test.expected),
strings.TrimSpace(
filtertest_test.RunFilter(t, test.input, test.filter))) {
t.FailNow()
}
})
}
}

View File

@@ -14,3 +14,5 @@ require (
sigs.k8s.io/kustomize/kyaml v0.13.0
sigs.k8s.io/yaml v1.2.0
)
replace sigs.k8s.io/kustomize/kyaml => ../kyaml

View File

@@ -223,8 +223,6 @@ k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8
k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE=
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e h1:KLHHjkdQFomZy8+06csTWZ0m1343QqxZhR2LJ1OxCYM=
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e/go.mod h1:vHXdDvt9+2spS2Rx9ql3I8tycm3H9FDfdUoIuKCefvw=
sigs.k8s.io/kustomize/kyaml v0.13.0 h1:9c+ETyNfSrVhxvphs+K2dzT3dh5oVPPEqPOE/cUpScY=
sigs.k8s.io/kustomize/kyaml v0.13.0/go.mod h1:FTJxEZ86ScK184NpGSAQcfEqee0nul8oLCK30D47m4E=
sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw=
sigs.k8s.io/yaml v1.2.0 h1:kr/MCeFWJWTwyaHoR9c8EjH9OumOmoF9YGiZd7lFm/Q=
sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc=

View File

@@ -521,7 +521,7 @@ func TestNameReferenceUnhappyRun(t *testing.T) {
},
}).ResMap(),
expectedErr: `updating name reference in 'rules/resourceNames' field of ` +
`'rbac.authorization.k8s.io_v1_ClusterRole|~X|cr'` +
`'ClusterRole.v1.rbac.authorization.k8s.io/cr.[noNs]'` +
`: considering field 'rules/resourceNames' of object
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole

View File

@@ -1,4 +1,4 @@
// Code generated by pluginator on PrefixSuffixTransformer; DO NOT EDIT.
// Code generated by pluginator on PrefixTransformer; DO NOT EDIT.
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins
@@ -6,32 +6,28 @@ package builtins
import (
"errors"
"sigs.k8s.io/kustomize/api/filters/prefixsuffix"
"sigs.k8s.io/kustomize/api/filters/prefix"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/yaml"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
// Add the given prefix and suffix to the field.
type PrefixSuffixTransformerPlugin struct {
// Add the given prefix to the field
type PrefixTransformerPlugin struct {
Prefix string `json:"prefix,omitempty" yaml:"prefix,omitempty"`
Suffix string `json:"suffix,omitempty" yaml:"suffix,omitempty"`
FieldSpecs types.FsSlice `json:"fieldSpecs,omitempty" yaml:"fieldSpecs,omitempty"`
}
// A Gvk skip list for prefix/suffix modification.
// hard coded for now - eventually should be part of config.
var prefixSuffixFieldSpecsToSkip = types.FsSlice{
var prefixFieldSpecsToSkip = types.FsSlice{
{Gvk: resid.Gvk{Kind: "CustomResourceDefinition"}},
{Gvk: resid.Gvk{Group: "apiregistration.k8s.io", Kind: "APIService"}},
{Gvk: resid.Gvk{Kind: "Namespace"}},
}
func (p *PrefixSuffixTransformerPlugin) Config(
func (p *PrefixTransformerPlugin) Config(
_ *resmap.PluginHelpers, c []byte) (err error) {
p.Prefix = ""
p.Suffix = ""
p.FieldSpecs = nil
err = yaml.Unmarshal(c, p)
if err != nil {
@@ -43,10 +39,10 @@ func (p *PrefixSuffixTransformerPlugin) Config(
return
}
func (p *PrefixSuffixTransformerPlugin) Transform(m resmap.ResMap) error {
// Even if both the Prefix and Suffix are empty we want
// to proceed with the transformation. This allows to add contextual
// information to the resources (AddNamePrefix and AddNameSuffix).
func (p *PrefixTransformerPlugin) Transform(m resmap.ResMap) error {
// Even if the Prefix is empty we want to proceed with the
// transformation. This allows to add contextual information
// to the resources (AddNamePrefix).
for _, r := range m.Resources() {
// TODO: move this test into the filter (i.e. make a better filter)
if p.shouldSkip(r.OrgId()) {
@@ -61,21 +57,21 @@ func (p *PrefixSuffixTransformerPlugin) Transform(m resmap.ResMap) error {
continue
}
// TODO: move this test into the filter.
if smellsLikeANameChange(&fs) {
if fs.Path == "metadata/name" {
// "metadata/name" is the only field.
// this will add a prefix and a suffix
// to the resource even if those are
// empty
// this will add a prefix to the resource
// even if it is empty
r.AddNamePrefix(p.Prefix)
r.AddNameSuffix(p.Suffix)
if p.Prefix != "" || p.Suffix != "" {
if p.Prefix != "" {
// TODO: There are multiple transformers that can change a resource's name, and each makes a call to
// StorePreviousID(). We should make it so that we only call StorePreviousID once per kustomization layer
// to avoid storing intermediate names between transformations, to prevent intermediate name conflicts.
r.StorePreviousId()
}
}
if err := r.ApplyFilter(prefixsuffix.Filter{
if err := r.ApplyFilter(prefix.Filter{
Prefix: p.Prefix,
Suffix: p.Suffix,
FieldSpec: fs,
}); err != nil {
return err
@@ -85,12 +81,8 @@ func (p *PrefixSuffixTransformerPlugin) Transform(m resmap.ResMap) error {
return nil
}
func smellsLikeANameChange(fs *types.FieldSpec) bool {
return fs.Path == "metadata/name"
}
func (p *PrefixSuffixTransformerPlugin) shouldSkip(id resid.ResId) bool {
for _, path := range prefixSuffixFieldSpecsToSkip {
func (p *PrefixTransformerPlugin) shouldSkip(id resid.ResId) bool {
for _, path := range prefixFieldSpecsToSkip {
if id.IsSelected(&path.Gvk) {
return true
}
@@ -98,6 +90,6 @@ func (p *PrefixSuffixTransformerPlugin) shouldSkip(id resid.ResId) bool {
return false
}
func NewPrefixSuffixTransformerPlugin() resmap.TransformerPlugin {
return &PrefixSuffixTransformerPlugin{}
func NewPrefixTransformerPlugin() resmap.TransformerPlugin {
return &PrefixTransformerPlugin{}
}

View File

@@ -0,0 +1,95 @@
// Code generated by pluginator on SuffixTransformer; DO NOT EDIT.
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins
import (
"errors"
"sigs.k8s.io/kustomize/api/filters/suffix"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
// Add the given suffix to the field
type SuffixTransformerPlugin struct {
Suffix string `json:"suffix,omitempty" yaml:"suffix,omitempty"`
FieldSpecs types.FsSlice `json:"fieldSpecs,omitempty" yaml:"fieldSpecs,omitempty"`
}
var suffixFieldSpecsToSkip = types.FsSlice{
{Gvk: resid.Gvk{Kind: "CustomResourceDefinition"}},
{Gvk: resid.Gvk{Group: "apiregistration.k8s.io", Kind: "APIService"}},
{Gvk: resid.Gvk{Kind: "Namespace"}},
}
func (p *SuffixTransformerPlugin) Config(
_ *resmap.PluginHelpers, c []byte) (err error) {
p.Suffix = ""
p.FieldSpecs = nil
err = yaml.Unmarshal(c, p)
if err != nil {
return
}
if p.FieldSpecs == nil {
return errors.New("fieldSpecs is not expected to be nil")
}
return
}
func (p *SuffixTransformerPlugin) Transform(m resmap.ResMap) error {
// Even if the Suffix is empty we want to proceed with the
// transformation. This allows to add contextual information
// to the resources (AddNameSuffix).
for _, r := range m.Resources() {
// TODO: move this test into the filter (i.e. make a better filter)
if p.shouldSkip(r.OrgId()) {
continue
}
id := r.OrgId()
// current default configuration contains
// only one entry: "metadata/name" with no GVK
for _, fs := range p.FieldSpecs {
// TODO: this is redundant to filter (but needed for now)
if !id.IsSelected(&fs.Gvk) {
continue
}
// TODO: move this test into the filter.
if fs.Path == "metadata/name" {
// "metadata/name" is the only field.
// this will add a suffix to the resource
// even if it is empty
r.AddNameSuffix(p.Suffix)
if p.Suffix != "" {
// TODO: There are multiple transformers that can change a resource's name, and each makes a call to
// StorePreviousID(). We should make it so that we only call StorePreviousID once per kustomization layer
// to avoid storing intermediate names between transformations, to prevent intermediate name conflicts.
r.StorePreviousId()
}
}
if err := r.ApplyFilter(suffix.Filter{
Suffix: p.Suffix,
FieldSpec: fs,
}); err != nil {
return err
}
}
}
return nil
}
func (p *SuffixTransformerPlugin) shouldSkip(id resid.ResId) bool {
for _, path := range suffixFieldSpecsToSkip {
if id.IsSelected(&path.Gvk) {
return true
}
}
return false
}
func NewSuffixTransformerPlugin() resmap.TransformerPlugin {
return &SuffixTransformerPlugin{}
}

View File

@@ -5,6 +5,7 @@ package generators
import (
"fmt"
"strings"
"github.com/go-errors/errors"
"sigs.k8s.io/kustomize/api/ifc"
@@ -50,7 +51,11 @@ func makeValidatedDataMap(
return nil, errors.Errorf(
"configmap %s illegally repeats the key `%s`", name, p.Key)
}
knownKeys[p.Key] = p.Value
lines := strings.Split(p.Value, "\n")
for i := range lines {
lines[i] = strings.TrimSuffix(lines[i], " ")
}
knownKeys[p.Key] = strings.Join(lines, "\n")
}
return knownKeys, nil
}

View File

@@ -21,16 +21,18 @@ func _() {
_ = x[PatchStrategicMergeTransformer-10]
_ = x[PatchTransformer-11]
_ = x[PrefixSuffixTransformer-12]
_ = x[ReplicaCountTransformer-13]
_ = x[SecretGenerator-14]
_ = x[ValueAddTransformer-15]
_ = x[HelmChartInflationGenerator-16]
_ = x[ReplacementTransformer-17]
_ = x[PrefixTransformer-13]
_ = x[SuffixTransformer-14]
_ = x[ReplicaCountTransformer-15]
_ = x[SecretGenerator-16]
_ = x[ValueAddTransformer-17]
_ = x[HelmChartInflationGenerator-18]
_ = x[ReplacementTransformer-19]
}
const _BuiltinPluginType_name = "UnknownAnnotationsTransformerConfigMapGeneratorIAMPolicyGeneratorHashTransformerImageTagTransformerLabelTransformerLegacyOrderTransformerNamespaceTransformerPatchJson6902TransformerPatchStrategicMergeTransformerPatchTransformerPrefixSuffixTransformerReplicaCountTransformerSecretGeneratorValueAddTransformerHelmChartInflationGeneratorReplacementTransformer"
const _BuiltinPluginType_name = "UnknownAnnotationsTransformerConfigMapGeneratorIAMPolicyGeneratorHashTransformerImageTagTransformerLabelTransformerLegacyOrderTransformerNamespaceTransformerPatchJson6902TransformerPatchStrategicMergeTransformerPatchTransformerPrefixSuffixTransformerPrefixTransformerSuffixTransformerReplicaCountTransformerSecretGeneratorValueAddTransformerHelmChartInflationGeneratorReplacementTransformer"
var _BuiltinPluginType_index = [...]uint16{0, 7, 29, 47, 65, 80, 99, 115, 137, 157, 181, 211, 227, 250, 273, 288, 307, 334, 356}
var _BuiltinPluginType_index = [...]uint16{0, 7, 29, 47, 65, 80, 99, 115, 137, 157, 181, 211, 227, 250, 267, 284, 307, 322, 341, 368, 390}
func (i BuiltinPluginType) String() string {
if i < 0 || i >= BuiltinPluginType(len(_BuiltinPluginType_index)-1) {

View File

@@ -4,7 +4,7 @@
package builtinhelpers
import (
"sigs.k8s.io/kustomize/api/builtins"
"sigs.k8s.io/kustomize/api/internal/builtins"
"sigs.k8s.io/kustomize/api/resmap"
)
@@ -25,6 +25,8 @@ const (
PatchStrategicMergeTransformer
PatchTransformer
PrefixSuffixTransformer
PrefixTransformer
SuffixTransformer
ReplicaCountTransformer
SecretGenerator
ValueAddTransformer
@@ -64,6 +66,35 @@ var GeneratorFactories = map[BuiltinPluginType]func() resmap.GeneratorPlugin{
HelmChartInflationGenerator: builtins.NewHelmChartInflationGeneratorPlugin,
}
type MultiTransformer struct {
transformers []resmap.TransformerPlugin
}
func (t *MultiTransformer) Transform(m resmap.ResMap) error {
for _, transformer := range t.transformers {
if err := transformer.Transform(m); err != nil {
return err
}
}
return nil
}
func (t *MultiTransformer) Config(h *resmap.PluginHelpers, b []byte) error {
for _, transformer := range t.transformers {
if err := transformer.Config(h, b); err != nil {
return err
}
}
return nil
}
func NewMultiTransformer() resmap.TransformerPlugin {
return &MultiTransformer{[]resmap.TransformerPlugin{
builtins.NewPrefixTransformerPlugin(),
builtins.NewSuffixTransformerPlugin(),
}}
}
var TransformerFactories = map[BuiltinPluginType]func() resmap.TransformerPlugin{
AnnotationsTransformer: builtins.NewAnnotationsTransformerPlugin,
HashTransformer: builtins.NewHashTransformerPlugin,
@@ -74,7 +105,9 @@ var TransformerFactories = map[BuiltinPluginType]func() resmap.TransformerPlugin
PatchJson6902Transformer: builtins.NewPatchJson6902TransformerPlugin,
PatchStrategicMergeTransformer: builtins.NewPatchStrategicMergeTransformerPlugin,
PatchTransformer: builtins.NewPatchTransformerPlugin,
PrefixSuffixTransformer: builtins.NewPrefixSuffixTransformerPlugin,
PrefixSuffixTransformer: NewMultiTransformer,
PrefixTransformer: builtins.NewPrefixTransformerPlugin,
SuffixTransformer: builtins.NewSuffixTransformerPlugin,
ReplacementTransformer: builtins.NewReplacementTransformerPlugin,
ReplicaCountTransformer: builtins.NewReplicaCountTransformerPlugin,
ValueAddTransformer: builtins.NewValueAddTransformerPlugin,

View File

@@ -11,9 +11,9 @@ import (
"github.com/pkg/errors"
"sigs.k8s.io/kustomize/api/builtins"
"sigs.k8s.io/kustomize/api/ifc"
"sigs.k8s.io/kustomize/api/internal/accumulator"
"sigs.k8s.io/kustomize/api/internal/builtins"
"sigs.k8s.io/kustomize/api/internal/plugins/builtinconfig"
"sigs.k8s.io/kustomize/api/internal/plugins/builtinhelpers"
"sigs.k8s.io/kustomize/api/internal/plugins/loader"
@@ -113,7 +113,11 @@ func (kt *KustTarget) MakeCustomizedResMap() (resmap.ResMap, error) {
}
func (kt *KustTarget) makeCustomizedResMap() (resmap.ResMap, error) {
ra, err := kt.AccumulateTarget(&resource.Origin{})
var origin *resource.Origin
if utils.StringSliceContains(kt.kustomization.BuildMetadata, types.OriginAnnotations) {
origin = &resource.Origin{}
}
ra, err := kt.AccumulateTarget(origin)
if err != nil {
return nil, err
}
@@ -371,7 +375,11 @@ func (kt *KustTarget) accumulateResources(
return nil, errors.Wrapf(
err, "accumulation err='%s'", errF.Error())
}
ra, err = kt.accumulateDirectory(ra, ldr, origin.Append(path), false)
if origin != nil {
ra, err = kt.accumulateDirectory(ra, ldr, origin.Append(path), false)
} else {
ra, err = kt.accumulateDirectory(ra, ldr, nil, false)
}
if err != nil {
return nil, errors.Wrapf(
err, "accumulation err='%s'", errF.Error())
@@ -392,8 +400,11 @@ func (kt *KustTarget) accumulateComponents(
return nil, fmt.Errorf("loader.New %q", errL)
}
var errD error
origin.Path = filepath.Join(origin.Path, path)
ra, errD = kt.accumulateDirectory(ra, ldr, origin, true)
if origin != nil {
ra, errD = kt.accumulateDirectory(ra, ldr, origin.Append(path), true)
} else {
ra, errD = kt.accumulateDirectory(ra, ldr, nil, true)
}
if errD != nil {
return nil, fmt.Errorf("accumulateDirectory: %q", errD)
}
@@ -459,12 +470,15 @@ func (kt *KustTarget) accumulateFile(
if err != nil {
return errors.Wrapf(err, "accumulating resources from '%s'", path)
}
if utils.StringSliceContains(kt.kustomization.BuildMetadata, "originAnnotations") {
origin = origin.Append(path)
err = resources.AnnotateAll(utils.OriginAnnotation, origin.String())
if origin != nil {
originAnno, err := origin.Append(path).String()
if err != nil {
return errors.Wrapf(err, "cannot add path annotation for '%s'", path)
}
err = resources.AnnotateAll(utils.OriginAnnotation, originAnno)
if err != nil || originAnno == "" {
return errors.Wrapf(err, "cannot add path annotation for '%s'", path)
}
}
err = ra.AppendAll(resources)
if err != nil {

View File

@@ -51,7 +51,8 @@ func (kt *KustTarget) configureBuiltinTransformers(
builtinhelpers.PatchStrategicMergeTransformer,
builtinhelpers.PatchTransformer,
builtinhelpers.NamespaceTransformer,
builtinhelpers.PrefixSuffixTransformer,
builtinhelpers.PrefixTransformer,
builtinhelpers.SuffixTransformer,
builtinhelpers.LabelTransformer,
builtinhelpers.AnnotationsTransformer,
builtinhelpers.PatchJson6902Transformer,
@@ -286,16 +287,14 @@ var transformerConfigurators = map[builtinhelpers.BuiltinPluginType]func(
result = append(result, p)
return
},
builtinhelpers.PrefixSuffixTransformer: func(
builtinhelpers.PrefixTransformer: func(
kt *KustTarget, bpt builtinhelpers.BuiltinPluginType, f tFactory, tc *builtinconfig.TransformerConfig) (
result []resmap.Transformer, err error) {
var c struct {
Prefix string
Suffix string
FieldSpecs []types.FieldSpec
Prefix string `json:"prefix,omitempty" yaml:"prefix,omitempty"`
FieldSpecs []types.FieldSpec `json:"fieldSpecs,omitempty" yaml:"fieldSpecs,omitempty"`
}
c.Prefix = kt.kustomization.NamePrefix
c.Suffix = kt.kustomization.NameSuffix
c.FieldSpecs = tc.NamePrefix
p := f()
err = kt.configureBuiltinPlugin(p, c, bpt)
@@ -305,6 +304,23 @@ var transformerConfigurators = map[builtinhelpers.BuiltinPluginType]func(
result = append(result, p)
return
},
builtinhelpers.SuffixTransformer: func(
kt *KustTarget, bpt builtinhelpers.BuiltinPluginType, f tFactory, tc *builtinconfig.TransformerConfig) (
result []resmap.Transformer, err error) {
var c struct {
Suffix string `json:"suffix,omitempty" yaml:"suffix,omitempty"`
FieldSpecs []types.FieldSpec `json:"fieldSpecs,omitempty" yaml:"fieldSpecs,omitempty"`
}
c.Suffix = kt.kustomization.NameSuffix
c.FieldSpecs = tc.NameSuffix
p := f()
err = kt.configureBuiltinPlugin(p, c, bpt)
if err != nil {
return nil, err
}
result = append(result, p)
return
},
builtinhelpers.ImageTagTransformer: func(
kt *KustTarget, bpt builtinhelpers.BuiltinPluginType, f tFactory, tc *builtinconfig.TransformerConfig) (
result []resmap.Transformer, err error) {

View File

@@ -11,6 +11,7 @@ import (
func GetDefaultFieldSpecs() []byte {
configData := [][]byte{
[]byte(namePrefixFieldSpecs),
[]byte(nameSuffixFieldSpecs),
[]byte(commonLabelFieldSpecs),
[]byte(commonAnnotationFieldSpecs),
[]byte(namespaceFieldSpecs),
@@ -27,6 +28,7 @@ func GetDefaultFieldSpecs() []byte {
func GetDefaultFieldSpecsAsMap() map[string]string {
result := make(map[string]string)
result["nameprefix"] = namePrefixFieldSpecs
result["namesuffix"] = nameSuffixFieldSpecs
result["commonlabels"] = commonLabelFieldSpecs
result["commonannotations"] = commonAnnotationFieldSpecs
result["namespace"] = namespaceFieldSpecs

View File

@@ -3,6 +3,7 @@
package builtinpluginconsts
// LINT.IfChange
const (
nameReferenceFieldSpecs = `
nameReference:
@@ -398,3 +399,5 @@ nameReference:
kind: Ingress
`
)
// LINT.ThenChange(/examples/transformerconfigs/README.md)

View File

@@ -0,0 +1,11 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package builtinpluginconsts
const (
nameSuffixFieldSpecs = `
nameSuffix:
- path: metadata/name
`
)

View File

@@ -292,8 +292,10 @@ metadata:
---
apiVersion: v1
data:
nonsense: "Lorem ipsum dolor sit amet, consectetur\nadipiscing elit, sed do eiusmod
tempor\nincididunt ut labore et dolore magna aliqua. \n"
nonsense: |
Lorem ipsum dolor sit amet, consectetur
adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua.
kind: ConfigMap
metadata:
annotations:
@@ -302,6 +304,6 @@ metadata:
app: mungebot
org: kubernetes
repo: test-infra
name: test-infra-app-config-49d6f5h7b5
name: test-infra-app-config-4thktg822m
`)
}

View File

@@ -368,7 +368,7 @@ resources:
t.Fatalf("Expected resource accumulation error")
}
if !strings.Contains(
err.Error(), "already registered id: apps_v1_StatefulSet|~X|my-sts") {
err.Error(), "already registered id: StatefulSet.v1.apps/my-sts.[noNs]") {
t.Fatalf("Unexpected err: %v", err)
}
}
@@ -459,7 +459,7 @@ resources:
t.Fatalf("Expected resource accumulation error")
}
if !strings.Contains(
err.Error(), "already registered id: apps_v1_StatefulSet|~X|my-sts") {
err.Error(), "already registered id: StatefulSet.v1.apps/my-sts.[noNs]") {
t.Fatalf("Unexpected err: %v", err)
}
}

View File

@@ -589,7 +589,7 @@ components:
- ../comp-b`),
},
runPath: "prod",
expectedError: "may not add resource with an already registered id: ~G_v1_Deployment|~X|proxy",
expectedError: "may not add resource with an already registered id: Deployment.v1.[noGrp]/proxy.[noNs]",
},
"components-cannot-add-the-same-base": {
input: []FileGen{writeTestBase,
@@ -608,7 +608,7 @@ components:
- ../comp-b`),
},
runPath: "prod",
expectedError: "may not add resource with an already registered id: ~G_v1_Deployment|~X|storefront",
expectedError: "may not add resource with an already registered id: Deployment.v1.[noGrp]/storefront.[noNs]",
},
"components-cannot-add-bases-containing-the-same-resource": {
input: []FileGen{writeTestBase,
@@ -639,7 +639,7 @@ components:
- ../comp-b`),
},
runPath: "prod",
expectedError: "may not add resource with an already registered id: ~G_v1_Deployment|~X|proxy",
expectedError: "may not add resource with an already registered id: Deployment.v1.[noGrp]/proxy.[noNs]",
},
}

View File

@@ -572,3 +572,74 @@ metadata:
name: test-k9cc55dfm5
`)
}
// regression test for https://github.com/kubernetes-sigs/kustomize/issues/4287
func TestMultilineDataEndsWithSpace(t *testing.T) {
th := kusttest_test.MakeHarness(t)
th.WriteK(".", `
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
configMapGenerator:
- name: config_bla
files:
- cfg.text=cfg.text
`)
th.WriteF("cfg.text", `bla
bla
bla
`)
m := th.Run(".", th.MakeDefaultOptions())
th.AssertActualEqualsExpected(
m, `apiVersion: v1
data:
cfg.text: |
bla
bla
bla
kind: ConfigMap
metadata:
name: config_bla-4k548khbf5
`)
}
// regression test to record the behavior prior to the fix for https://github.com/kubernetes-sigs/kustomize/issues/4287
// to ensure that the fix does not affect leading and trailing newlines in ConfigMap data
func TestMultilineDataEndsLeadingAndTrailingNewlines(t *testing.T) {
th := kusttest_test.MakeHarness(t)
th.WriteK(".", `
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
configMapGenerator:
- name: config_bla
files:
- cfg.text=cfg.text
`)
th.WriteF("cfg.text", `
bla
bla
bla
`)
m := th.Run(".", th.MakeDefaultOptions())
th.AssertActualEqualsExpected(
m, `apiVersion: v1
data:
cfg.text: |2+
bla
bla
bla
kind: ConfigMap
metadata:
name: config_bla-8dm6c68g22
`)
}

View File

@@ -6,6 +6,8 @@ package krusty_test
import (
"testing"
"sigs.k8s.io/kustomize/api/types"
kusttest_test "sigs.k8s.io/kustomize/api/testutils/kusttest"
)
@@ -13,8 +15,8 @@ import (
// This is a NamePrefixer that touches Deployments
// and Services exclusively.
func TestCustomNamePrefixer(t *testing.T) {
th := kusttest_test.MakeEnhancedHarness(t).
PrepBuiltin("PrefixSuffixTransformer")
th := kusttest_test.MakeEnhancedHarness(t)
th.GetPluginConfig().BpLoadingOptions = types.BploUseStaticallyLinked
defer th.Reset()
th.WriteK(".", `

View File

@@ -7,6 +7,7 @@ import (
"testing"
kusttest_test "sigs.k8s.io/kustomize/api/testutils/kusttest"
"sigs.k8s.io/kustomize/api/types"
)
// Demo custom configuration as a base.
@@ -15,9 +16,9 @@ import (
// kustomizations.
func TestReusableCustomTransformers(t *testing.T) {
th := kusttest_test.MakeEnhancedHarness(t).
PrepBuiltin("PrefixSuffixTransformer").
PrepBuiltin("AnnotationsTransformer").
PrepBuiltin("LabelTransformer")
th.GetPluginConfig().BpLoadingOptions = types.BploUseStaticallyLinked
defer th.Reset()
// First write three custom configurations for builtin plugins.

View File

@@ -141,7 +141,7 @@ resources:
t.Fatalf("Expected resource accumulation error")
}
if !strings.Contains(
err.Error(), "already registered id: apps_v1_Deployment|~X|my-deployment") {
err.Error(), "already registered id: Deployment.v1.apps/my-deployment.[noNs]") {
t.Fatalf("Unexpected err: %v", err)
}
}

View File

@@ -7,7 +7,7 @@ import (
"fmt"
"path/filepath"
"sigs.k8s.io/kustomize/api/builtins"
"sigs.k8s.io/kustomize/api/internal/builtins"
pLdr "sigs.k8s.io/kustomize/api/internal/plugins/loader"
"sigs.k8s.io/kustomize/api/internal/target"
"sigs.k8s.io/kustomize/api/konfig"

View File

@@ -0,0 +1,41 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package krusty_test
import (
"testing"
kusttest_test "sigs.k8s.io/kustomize/api/testutils/kusttest"
)
func TestLegacyPrefixSuffixTransformer(t *testing.T) {
th := kusttest_test.MakeHarness(t)
th.WriteK(".", `
resources:
- service.yaml
transformers:
- |-
apiVersion: builtin
kind: PrefixSuffixTransformer
metadata:
name: notImportantHere
prefix: baked-
suffix: -pie
fieldSpecs:
- path: metadata/name
`)
th.WriteF("service.yaml", `
apiVersion: v1
kind: Service
metadata:
name: apple
`)
m := th.Run(".", th.MakeDefaultOptions())
th.AssertActualEqualsExpected(m, `
apiVersion: v1
kind: Service
metadata:
name: baked-apple-pie
`)
}

View File

@@ -531,3 +531,58 @@ metadata:
name: secret-example-7hf4fh868h
`)
}
func TestUnrelatedNameReferenceReplacement_Issue4254_Issue3418(t *testing.T) {
th := kusttest_test.MakeHarness(t)
// The cluster-autoscaler lease name should not be changed.
th.WriteF("role.yaml", `
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cluster-autoscaler
rules:
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
resourceNames: ["cluster-autoscaler"]
verbs: ["get","update"]
`)
th.WriteK(".", `
resources:
- role.yaml
configMapGenerator:
- name: cluster-autoscaler
namespace: kube-system
literals:
- AWS_REGION="us-east-1"
`)
// The resourceNames for the leases resource in the ClusterRole should NOT be
// updated with the name suffix, because it's not targeting the generated
// configmap. The value at rules[0].resourceNames[0] is currently incorrect.
m := th.Run(".", th.MakeDefaultOptions())
th.AssertActualEqualsExpected(m, `
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cluster-autoscaler
rules:
- apiGroups:
- coordination.k8s.io
resourceNames:
- cluster-autoscaler-h8mmcct52k
resources:
- leases
verbs:
- get
- update
---
apiVersion: v1
data:
AWS_REGION: us-east-1
kind: ConfigMap
metadata:
name: cluster-autoscaler-h8mmcct52k
namespace: kube-system
`)
}

View File

@@ -128,7 +128,7 @@ spec:
err := th.RunWithErr("mango", th.MakeDefaultOptions())
if !strings.Contains(
err.Error(), "multiple matches for Id apps_v1_Deployment|~X|banana; failed to find unique target for patch") {
err.Error(), "multiple matches for Id Deployment.v1.apps/banana.[noNs]; failed to find unique target for patch Deployment.v1.apps/banana.[noNs]") {
t.Fatalf("Unexpected err: %v", err)
}

View File

@@ -215,7 +215,7 @@ func (m *resWrangler) GetById(
if err != nil {
return nil, fmt.Errorf(
"%s; failed to find unique target for patch %s",
err.Error(), id.GvknString())
err.Error(), id.String())
}
return r, nil
}

View File

@@ -8,6 +8,7 @@ import (
"strings"
"sigs.k8s.io/kustomize/api/internal/git"
kyaml "sigs.k8s.io/kustomize/kyaml/yaml"
)
// Origin retains information about where resources in the output
@@ -15,19 +16,22 @@ import (
type Origin struct {
// Path is the path to the resource, rooted from the directory upon
// which `kustomize build` was invoked
Path string
Path string `json:"path,omitempty" yaml:"path,omitempty"`
// Repo is the remote repository that the resource originated from if it is
// not from a local file
Repo string
Repo string `json:"repo,omitempty" yaml:"repo,omitempty"`
// Ref is the ref of the remote repository that the resource originated from
// if it is not from a local file
Ref string
Ref string `json:"ref,omitempty" yaml:"ref,omitempty"`
}
// Copy returns a copy of origin
func (origin *Origin) Copy() Origin {
if origin == nil {
return Origin{}
}
return *origin
}
@@ -47,14 +51,7 @@ func (origin *Origin) Append(path string) *Origin {
}
// String returns a string version of origin
func (origin *Origin) String() string {
var anno string
anno = anno + "path: " + origin.Path + "\n"
if origin.Repo != "" {
anno = anno + "repo: " + origin.Repo + "\n"
}
if origin.Ref != "" {
anno = anno + "ref: " + origin.Ref + "\n"
}
return anno
func (origin *Origin) String() (string, error) {
anno, err := kyaml.Marshal(origin)
return string(anno), err
}

View File

@@ -6,6 +6,7 @@ package resource_test
import (
"testing"
"github.com/stretchr/testify/assert"
. "sigs.k8s.io/kustomize/api/resource"
)
@@ -34,10 +35,9 @@ repo: https://github.com/kubernetes-sigs/kustomize
},
}
for _, test := range tests {
actual := test.in.Append(test.path).String()
if actual != test.expected {
t.Fatalf("Expected %v, but got %v\n", test.expected, actual)
}
actual, err := test.in.Append(test.path).String()
assert.NoError(t, err)
assert.Equal(t, actual, test.expected)
}
}
@@ -76,8 +76,8 @@ repo: github.com/kubernetes-sigs/kustomize/examples/multibases/dev/
}
for _, test := range tests {
if test.in.String() != test.expected {
t.Fatalf("Expected %v, but got %v\n", test.expected, test.in.String())
}
actual, err := test.in.String()
assert.NoError(t, err)
assert.Equal(t, actual, test.expected)
}
}

View File

@@ -42,9 +42,12 @@ var BuildAnnotations = []string{
kioutil.PathAnnotation,
kioutil.IndexAnnotation,
kioutil.SeqIndentAnnotation,
kioutil.IdAnnotation,
kioutil.InternalAnnotationsMigrationResourceIDAnnotation,
kioutil.LegacyPathAnnotation,
kioutil.LegacyIndexAnnotation,
kioutil.LegacyIdAnnotation,
}
func (r *Resource) ResetRNode(incoming *Resource) {

View File

@@ -1377,7 +1377,7 @@ spec:
numReplicas: 1
`))
assert.NoError(t, err)
r.SetGvk(resid.GvkFromString("grp_ver_knd"))
r.SetGvk(resid.GvkFromString("knd.ver.grp"))
gvk := r.GetGvk()
if expected, actual := "grp", gvk.Group; expected != actual {
t.Fatalf("expected '%s', got '%s'", expected, actual)
@@ -1400,30 +1400,30 @@ spec:
numReplicas: 1
`))
assert.NoError(t, err)
r.AppendRefBy(resid.FromString("gr1_ver1_knd1|ns1|name1"))
r.AppendRefBy(resid.FromString("knd1.ver1.gr1/name1.ns1"))
assert.Equal(t, r.RNode.MustString(), `apiVersion: v1
kind: Deployment
metadata:
name: clown
annotations:
internal.config.kubernetes.io/refBy: gr1_ver1_knd1|ns1|name1
internal.config.kubernetes.io/refBy: knd1.ver1.gr1/name1.ns1
spec:
numReplicas: 1
`)
assert.Equal(t, r.GetRefBy(), []resid.ResId{resid.FromString("gr1_ver1_knd1|ns1|name1")})
assert.Equal(t, r.GetRefBy(), []resid.ResId{resid.FromString("knd1.ver1.gr1/name1.ns1")})
r.AppendRefBy(resid.FromString("gr2_ver2_knd2|ns2|name2"))
r.AppendRefBy(resid.FromString("knd2.ver2.gr2/name2.ns2"))
assert.Equal(t, r.RNode.MustString(), `apiVersion: v1
kind: Deployment
metadata:
name: clown
annotations:
internal.config.kubernetes.io/refBy: gr1_ver1_knd1|ns1|name1,gr2_ver2_knd2|ns2|name2
internal.config.kubernetes.io/refBy: knd1.ver1.gr1/name1.ns1,knd2.ver2.gr2/name2.ns2
spec:
numReplicas: 1
`)
assert.Equal(t, r.GetRefBy(), []resid.ResId{
resid.FromString("gr1_ver1_knd1|ns1|name1"),
resid.FromString("gr2_ver2_knd2|ns2|name2"),
resid.FromString("knd1.ver1.gr1/name1.ns1"),
resid.FromString("knd2.ver2.gr2/name2.ns2"),
})
}

View File

@@ -17,6 +17,7 @@ const (
ComponentVersion = "kustomize.config.k8s.io/v1alpha1"
ComponentKind = "Component"
MetadataNamespacePath = "metadata/namespace"
OriginAnnotations = "originAnnotations"
)
// Kustomization holds the information needed to generate customized k8s api resources.

View File

@@ -18,3 +18,5 @@ require (
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e
sigs.k8s.io/kustomize/kyaml v0.13.0
)
replace sigs.k8s.io/kustomize/kyaml => ../../kyaml

View File

@@ -245,8 +245,6 @@ k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8
k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE=
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e h1:KLHHjkdQFomZy8+06csTWZ0m1343QqxZhR2LJ1OxCYM=
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e/go.mod h1:vHXdDvt9+2spS2Rx9ql3I8tycm3H9FDfdUoIuKCefvw=
sigs.k8s.io/kustomize/kyaml v0.13.0 h1:9c+ETyNfSrVhxvphs+K2dzT3dh5oVPPEqPOE/cUpScY=
sigs.k8s.io/kustomize/kyaml v0.13.0/go.mod h1:FTJxEZ86ScK184NpGSAQcfEqee0nul8oLCK30D47m4E=
sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw=
sigs.k8s.io/yaml v1.2.0 h1:kr/MCeFWJWTwyaHoR9c8EjH9OumOmoF9YGiZd7lFm/Q=
sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc=

View File

@@ -293,6 +293,8 @@ apiVersion: apps/v1
kind: Deployment
metadata:
name: foo
annotations:
internal.config.kubernetes.io/annotations-migration-resource-id: '1'
`,
}
},

View File

@@ -6,8 +6,10 @@ require (
github.com/rakyll/statik v0.1.7
github.com/spf13/cobra v1.0.0
github.com/stretchr/testify v1.5.1
sigs.k8s.io/kustomize/api v0.10.0
sigs.k8s.io/kustomize/api v0.10.1
sigs.k8s.io/kustomize/kyaml v0.13.0
)
replace sigs.k8s.io/kustomize/api => ../../api
replace sigs.k8s.io/kustomize/kyaml => ../../kyaml

View File

@@ -228,8 +228,6 @@ k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8
k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE=
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e h1:KLHHjkdQFomZy8+06csTWZ0m1343QqxZhR2LJ1OxCYM=
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e/go.mod h1:vHXdDvt9+2spS2Rx9ql3I8tycm3H9FDfdUoIuKCefvw=
sigs.k8s.io/kustomize/kyaml v0.13.0 h1:9c+ETyNfSrVhxvphs+K2dzT3dh5oVPPEqPOE/cUpScY=
sigs.k8s.io/kustomize/kyaml v0.13.0/go.mod h1:FTJxEZ86ScK184NpGSAQcfEqee0nul8oLCK30D47m4E=
sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw=
sigs.k8s.io/yaml v1.2.0 h1:kr/MCeFWJWTwyaHoR9c8EjH9OumOmoF9YGiZd7lFm/Q=
sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc=

View File

@@ -141,7 +141,7 @@ func newWriter(r string) (*writer, error) {
// $HOME/kustomize/api/builtins
func makeOutputFileName(root string) string {
return filepath.Join(
"..", "..", "..", "api", packageForGeneratedCode, root+".go")
"..", "..", "..", "api/internal", packageForGeneratedCode, root+".go")
}
func (w *writer) Close() error {

View File

@@ -1,8 +0,0 @@
module sigs.k8s.io/kustomize/cmd/prchecker
go 1.16
require (
github.com/google/go-github v17.0.0+incompatible
github.com/google/go-querystring v1.0.0 // indirect
)

View File

@@ -1,4 +0,0 @@
github.com/google/go-github v17.0.0+incompatible h1:N0LgJ1j65A7kfXrZnUDaYCs/Sf4rEjNlfyDHW9dolSY=
github.com/google/go-github v17.0.0+incompatible/go.mod h1:zLgOLi98H3fifZn+44m+umXrS52loVEgC2AApnigrVQ=
github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=

View File

@@ -1,271 +0,0 @@
// prchecker examines pull requests
//
// - When a PR includes files from multiple modules that we'd rather not
// modify at the same time (in an effort to have more self-contained
// release notes), the script will exit with a non-zero exit code.
//
// Usage:
//
// go run . \
// -owner=kubernetes-sigs \
// -repo=kustomize \
// -pr=2997 \
// cmd/config api kustomize kyaml
package main
import (
"context"
"flag"
"fmt"
"os"
"regexp"
"strings"
"github.com/google/go-github/github"
)
// splitCommitsHelp is a help message displayed when a PR has commits which
// span modules.
const splitCommitsHelp = "\nCommits that include multiple Go modules must be split into multiple commits that each touch no more than one module.\n" +
"Splitting instructions: https://git-scm.com/docs/git-rebase#_splitting_commits\n"
// Ignore span pattern defines a regular expression pattern that, when matched,
// causes this check to be ignored.
// Pattern expects "ALLOW_MODULE_SPAN" in CAPS on a line by itself.
// Spaces may be included before or after but no other characters with one
// exception. ">" may be used to quote the exclusion.
// Pattern may be provided at any point in the pull request description.
//
// Ex: "ALLOW_MODULE_SPAN", "> ALLOW_MODULE_SPAN", " ALLOW_MODULE_SPAN "
const ignoreSpanPattern = "(?m)^>?\\s*ALLOW_MODULE_SPAN\\s*$"
// Changeset represents a set of file modifications associated with a commit
type Changeset struct {
files []string
id string
}
// GitHubRepository represents a pairing of the owner and repository to make
// passing around references to specific projects simpler
type GitHubRepository struct {
client *github.Client
owner *string
repo *string
}
// GitHubService is the collection of GitHub API interactions this service consumes
type GitHubService interface {
GetPullRequest(prId int) (*github.PullRequest, *github.Response, error)
GetCommit(commitSha string) (*github.RepositoryCommit, *github.Response, error)
ListCommits(prId int, options *github.ListOptions) ([]*github.RepositoryCommit, *github.Response, error)
}
// GetPullRequest retrieves details about a pull request from GitHub API
func (repository GitHubRepository) GetPullRequest(prId int) (
*github.PullRequest, *github.Response, error) {
return repository.client.PullRequests.Get(
context.Background(),
*repository.owner,
*repository.repo,
prId)
}
// GetCommit retrieves commit details from GitHub API
func (repository GitHubRepository) GetCommit(commitSha string) (
*github.RepositoryCommit, *github.Response, error) {
return repository.client.Repositories.GetCommit(context.Background(),
*repository.owner,
*repository.repo,
commitSha)
}
// ListCommits lists commits in a PR via GitHub API
func (repository GitHubRepository) ListCommits(prId int, options *github.ListOptions) (
[]*github.RepositoryCommit, *github.Response, error) {
return repository.client.PullRequests.ListCommits(context.Background(),
*repository.owner,
*repository.repo,
prId,
options)
}
func main() {
owner := flag.String("owner", "", "the github repository owner name")
repo := flag.String("repo", "", "the github repository name")
pullrequest := flag.Int("pr", -1, "the pull request number")
flag.Parse()
// Treat all following arguments as restricted directories
restrictedPaths := flag.Args()
// Short circuit check if restricted paths is less than 2
// Conflicts won't exist in this scenario so we don't need to call the API
if len(restrictedPaths) <= 1 {
fmt.Println("Check not run. Add at least two restricted paths and run again.")
os.Exit(0)
}
client := github.NewClient(nil)
githubRepo := &GitHubRepository{
client: client,
owner: owner,
repo: repo,
}
// Check if module span is allowed before scanning on commits
isSpanAllowed, err := ModuleSpanAllowed(githubRepo, *pullrequest)
if err != nil {
fmt.Printf("unable to retrieve pull request details: %v", err.Error())
os.Exit(2)
}
if isSpanAllowed {
fmt.Println("Check not run. Module spanning was allowed in this Pull Request.")
os.Exit(0)
}
isSpanningPull, _, err := PullRequestSpanningPathList(githubRepo, *pullrequest, restrictedPaths)
if err != nil {
fmt.Printf("unable to retrieve pull request details: %v", err)
os.Exit(2)
}
// Exit with error if two or more restricted directories where modified
if isSpanningPull {
// Provide a suggestion for potential solution if the check fails.
fmt.Println(splitCommitsHelp)
os.Exit(1)
}
}
// ConstructChangeset creates a changeset from a GitHub Commit object
func ConstructChangeset(commit *github.RepositoryCommit) *Changeset {
id := commit.SHA
fileset := []string{}
for _, file := range commit.Files {
fileset = append(fileset, *file.Filename)
}
return &Changeset{
files: fileset,
id: *id,
}
}
// StringAllowsModuleSpan tests if a string matches the allow span regex
func StringAllowsModuleSpan(body string) (bool, error) {
return regexp.MatchString(ignoreSpanPattern, body)
}
// ModuleSpanAllowed tests a Pull Requests description for a regular
// expression. If the expression matches then spanning modules are allowed.
func ModuleSpanAllowed(repository GitHubService, pullId int) (bool, error) {
// Note: There are multiple ways to pull a github commit object
// we want a RepositoryCommit.
pullRequest, _, err := repository.GetPullRequest(pullId)
if err != nil {
return false, err
}
return StringAllowsModuleSpan(*pullRequest.Body)
}
// GetCommitChanges looks up a github commit by SHA and returns a Changeset
// containing the modified files in the specified commit.
func GetCommitChanges(repository GitHubService, commitSha string) (*Changeset, error) {
commit, _, err := repository.GetCommit(commitSha)
if err != nil {
return nil, err
}
return ConstructChangeset(commit), nil
}
// GetPullRequestCommits constructs a list of all commits and the associated
// files changes in the given pull request
func GetPullRequestCommits(repository GitHubService, pullrequest int) ([]*Changeset, error) {
// foundFiles across all pages from github api
var collectedCommits []*github.RepositoryCommit
// Github only returns a limited set of commits per request and PR's may
// exceed this so loop until all pages have been enumerated.
options := &github.ListOptions{Page: 1}
for options.Page != 0 {
commits, response, err := repository.ListCommits(pullrequest, options)
// If an error has occurred while querying api exit early, report error
if err != nil {
return nil, err
}
collectedCommits = append(collectedCommits, commits...)
// setup next page to continue loop
options = &github.ListOptions{Page: response.NextPage}
}
var changesetResults []*Changeset
for _, commit := range collectedCommits {
// The repository commits from list commits are not hydrated
// We will need to retrieve the complete object:
changeset, err := GetCommitChanges(repository, *commit.SHA)
if err != nil {
return nil, err
}
changesetResults = append(changesetResults, changeset)
}
return changesetResults, nil
}
// PullRequestSpanningPathList tests if a pull request spans multiple
// directory paths
func PullRequestSpanningPathList(repository GitHubService, pullrequest int, paths []string) (bool, []*Changeset, error) {
// Create a buffer for commits
changesets, err := GetPullRequestCommits(repository, pullrequest)
if err != nil {
return false, nil, err
}
spanningChangesExist := false
for _, changeset := range changesets {
if changeset.isSpanningPaths(paths) {
// When detecting the first spanning changeset print a prefix message
if !spanningChangesExist {
fmt.Printf("Spanning changesets detected in the following commits:\n\n")
}
fmt.Printf("\t* %s\n", changeset.id)
// In order provide a full list of outstanding commits, do not shortcircuit this check
spanningChangesExist = true
}
}
return spanningChangesExist, changesets, nil
}
// isSpanningPaths tests if a changeset is spanning
// multiple directory paths.
func (changeset *Changeset) isSpanningPaths(paths []string) bool {
matchedPath := ""
for _, file := range changeset.files {
for _, path := range paths {
if strings.HasPrefix(file, path) {
// If a different path has already matched then the changeset spans multiple restricted paths
if matchedPath != "" && matchedPath != path {
return true
}
matchedPath = path
}
}
}
return false
}

View File

@@ -1,303 +0,0 @@
package main
import (
"testing"
"github.com/google/go-github/github"
)
func TestStringAllowsModuleSpan(t *testing.T) {
var tests = []struct {
name string
body string
matchExpected bool
}{
{
"exception not included",
"foo",
false,
},
{
"exception mentioned in sentence does not exempt span check",
"don't ALLOW_MODULE_SPAN",
false,
},
{
"PR body is just exception",
"ALLOW_MODULE_SPAN",
true,
},
{
"support markdown quoting exception",
"> ALLOW_MODULE_SPAN",
true,
},
{
"support whitespace padding",
"\t ALLOW_MODULE_SPAN\t ",
true,
},
{
"module span exemption allowed at start of string",
"ALLOW_MODULE_SPAN\nat start of file",
true,
},
{
"module span exemption allowed at end of string",
"at end of file\nALLOW_MODULE_SPAN",
true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result, err := StringAllowsModuleSpan(tt.body)
if err != nil {
t.Errorf(err.Error())
}
if result != tt.matchExpected {
t.Errorf("got %t, want %t", result, tt.matchExpected)
}
})
}
}
func TestIsModuleSpanAllowed(t *testing.T) {
var tests = []struct {
name string
body string
matchExpected bool
}{
{
"module spanning not allowed",
"don't ALLOW_MODULE_SPAN",
false,
},
{
"module spanning allowed",
"ALLOW_MODULE_SPAN",
true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
client := FakeGitHubService{
pullRequest: &github.PullRequest{
Body: &tt.body,
},
}
result, _ := ModuleSpanAllowed(&client, 1)
if result != tt.matchExpected {
t.Errorf("got %t, want %t", result, tt.matchExpected)
}
})
}
}
func TestGetCommitChanges(t *testing.T) {
client := FakeGitHubService{
commit: &github.RepositoryCommit{
SHA: github.String("abc123"),
Files: []github.CommitFile{{Filename: github.String("foo")}, {Filename: github.String("bar")}},
},
}
result, _ := GetCommitChanges(&client, "abc123")
if result.id != "abc123" {
t.Errorf("got %v, want %v", result.id, "abc123")
}
if len(result.files) != 2 {
t.Errorf("got %v, want %v", len(result.files), "2")
}
expectedFiles := []string{"foo", "bar"}
if !StringSliceAreEqual(result.files, expectedFiles) {
t.Errorf("got %v, want %v", result.files, expectedFiles)
}
}
func TestGetPullRequestCommits(t *testing.T) {
client := FakeGitHubService{
commit: &github.RepositoryCommit{
SHA: github.String("abc123"),
Files: []github.CommitFile{{Filename: github.String("foo")}, {Filename: github.String("bar")}},
},
commitList: []*github.RepositoryCommit{
{
SHA: github.String("abc123"),
},
{
SHA: github.String("abc123"),
},
},
}
result, _ := GetPullRequestCommits(&client, 42)
if len(result) != 2 {
t.Errorf("got %v, want %v", len(result), 2)
}
expectedFiles := []string{"foo", "bar"}
if !StringSliceAreEqual(result[0].files, expectedFiles) {
t.Errorf("[%d] got %v, want %v", 0, result[0].files, expectedFiles)
}
if !StringSliceAreEqual(result[1].files, expectedFiles) {
t.Errorf("[%d] got %v, want %v", 1, result[1].files, expectedFiles)
}
}
func TestContstructingChangeset(t *testing.T) {
var tests = []struct {
name string
sha string
files []github.CommitFile
expected Changeset
}{
{
"construct from single file",
"abc123",
[]github.CommitFile{{Filename: github.String("foo")}},
Changeset{id: "abc123", files: []string{"foo"}},
},
{
"construct from multiple files",
"1234",
[]github.CommitFile{{Filename: github.String("foo")}, {Filename: github.String("bar")}},
Changeset{id: "1234", files: []string{"foo", "bar"}},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
commit := &github.RepositoryCommit{SHA: github.String(tt.sha), Files: tt.files}
result := ConstructChangeset(commit)
if tt.expected.id != result.id {
t.Errorf("got %v, want %v", result.id, tt.expected.id)
}
if !StringSliceAreEqual(tt.expected.files, result.files) {
t.Errorf("got %v, want %v", result.files, tt.expected.files)
}
})
}
}
func TestIsChangesetSpanning(t *testing.T) {
var tests = []struct {
name string
changeset []string
files []string
expected bool
}{
{
"matching sets of 1 element do not span",
[]string{"1"},
[]string{"1"},
false,
},
{
"subdirectories do not match top level directories",
[]string{"a/1"},
[]string{"1"},
false,
},
{
"distinct sets do not span",
[]string{"1", "2"},
[]string{"a", "b"},
false,
},
{
"single matching path does not span",
[]string{"1", "a"},
[]string{"1", "2"},
false,
},
{
"path and subdirectory of same restriction do not span",
[]string{"1", "1/a"},
[]string{"1", "2", "3"},
false,
},
{
"matching sets span",
[]string{"1", "2"},
[]string{"1", "2"},
true,
},
{
"superset of restricted paths spans",
[]string{"1", "2", "3"},
[]string{"1", "2"},
true,
},
{
"subset of restricted paths spans",
[]string{"1", "3"},
[]string{"1", "2", "3"},
true,
},
{
"subdirectories of restricted paths span",
[]string{"1/a", "3/b"},
[]string{"1", "2", "3"},
true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
changeset := &Changeset{files: tt.changeset}
result := changeset.isSpanningPaths(tt.files)
if result != tt.expected {
t.Errorf("got %t, want %t", result, tt.expected)
}
})
}
}
type FakeGitHubService struct {
pullRequest *github.PullRequest
commit *github.RepositoryCommit
commitList []*github.RepositoryCommit
}
func (repository FakeGitHubService) GetPullRequest(prId int) (*github.PullRequest, *github.Response, error) {
return repository.pullRequest, nil, nil
}
func (repository FakeGitHubService) GetCommit(commitSha string) (*github.RepositoryCommit, *github.Response, error) {
return repository.commit, nil, nil
}
func (repository FakeGitHubService) ListCommits(prId int, options *github.ListOptions) ([]*github.RepositoryCommit, *github.Response, error) {
return repository.commitList, &github.Response{NextPage: 0}, nil
}
func StringSliceAreEqual(left []string, right []string) bool {
if len(left) != len(right) {
return false
}
for i, elem := range left {
if elem != right[i] {
return false
}
}
return true
}

View File

@@ -1,2 +1,2 @@
Moved to [https://kubernetes-sigs.github.io/kustomize](https://kubectl.docs.kubernetes.io/guides/extending_kustomize/)
Moved to [https://kubectl.docs.kubernetes.io/guides/extending_kustomize/](https://kubectl.docs.kubernetes.io/guides/extending_kustomize/)

View File

@@ -1,2 +1,2 @@
Moved to [https://kubernetes-sigs.github.io/kustomize](https://kubernetes-sigs.github.io/kustomize/guides/plugins/builtins/)
Moved to [https://kubectl.docs.kubernetes.io/guides/extending_kustomize/](https://kubectl.docs.kubernetes.io/guides/extending_kustomize/)

View File

@@ -1,2 +1,2 @@
Moved to [https://kubernetes-sigs.github.io/kustomize](https://kubernetes-sigs.github.io/kustomize/guides/plugins)
Moved to [https://kubectl.docs.kubernetes.io/guides/extending_kustomize/](https://kubectl.docs.kubernetes.io/guides/extending_kustomize/)

View File

@@ -1,2 +1,2 @@
Moved to [https://kubernetes-sigs.github.io/kustomize](https://kubernetes-sigs.github.io/kustomize/guides/plugins)
Moved to [https://kubectl.docs.kubernetes.io/guides/extending_kustomize/](https://kubectl.docs.kubernetes.io/guides/extending_kustomize/)

View File

@@ -1,2 +1,2 @@
Moved to [https://kubernetes-sigs.github.io/kustomize](https://kubernetes-sigs.github.io/kustomize/guides/plugins)
Moved to [https://kubectl.docs.kubernetes.io/guides/extending_kustomize/](https://kubectl.docs.kubernetes.io/guides/extending_kustomize/)

View File

@@ -119,44 +119,472 @@ commonAnnotations:
## Name reference transformer
Name reference transformer's configuration is different from all other transformers. It contains a list of `nameReferences`, which represent all of the possible fields that a type could be used as a reference in other types of resources. A `nameReference` contains a type such as ConfigMap as well as a list of `fieldSpecs` where ConfigMap is referenced in other resources. Here is an example:
`nameReference` Transformer is used to tie a target resource's name to a list of other resources' referrers' names.
Once tied, the referrers' names will change alongside the target name via transformers like `namePrefix` and `nameSuffix`
### Usage
- The syntax `nameReference` should be written in the `configurations` files, not directly in `kustomization.yaml`
- kustomize has a set of builtin nameReference, and you don't need to write additional configs to use those nameReference. Check the full list [here](#builtin-namereference).
- The referrer's `name` has to match the target's `name`. Otherwise, the referrer's name won't be changed. This name can be the target's current name, or, if the target has been through multiple name transformations, can be any of the target's previous names.
- `nameReference` should be used together with other name-changing transformers. Using it alone won't make any changes.
### Example
kustomization.yaml
```yaml
kind: ConfigMap
version: v1
fieldSpecs:
- kind: Pod
version: v1
path: spec/volumes/configMap/name
- kind: Deployment
path: spec/template/spec/volumes/configMap/name
- kind: Job
path: spec/template/spec/volumes/configMap/name
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
# resources.yaml contains a Gorilla and a AnimalPark
- resources.yaml
# Add name prefix "sample-" to the Gorilla's and AnimalPark's name
namePrefix: sample-
configurations:
# Tie target Gorilla to AnimalPark's referrer spec.gorillaRef.
# Once Gorilla name is changed, the AnimalPark referrerd Gorilla name will be changed as well.
- nameReference.yaml
```
nameReference.yaml
```yaml
nameReference:
- kind: Gorilla
fieldSpecs:
- kind: AnimalPark
path: spec/gorillaRef/name
```
Name reference transformer's configuration contains a list of `nameReferences` for resources such as ConfigMap, Secret, Service, Role, and ServiceAccount. Here is an example configuration:
resources.yaml
```yaml
apiVersion: animal/v1
kind: Gorilla
metadata:
name: gg
---
apiVersion: animal/v1
kind: AnimalPark
metadata:
name: ap
spec:
gorillaRef:
name: gg
kind: Gorilla
apiVersion: animal/v1
```
Output of `kustomize build`
```yaml
apiVersion: animal/v1
kind: AnimalPark
metadata:
name: sample-ap # changed by `namePrefix`
spec:
gorillaRef:
apiVersion: animal/v1
kind: Gorilla
name: sample-gg # changed by `nameReference`
---
apiVersion: animal/v1
kind: Gorilla
metadata:
name: sample-gg # changed by `namePrefix`
```
### builtin NameReference
```yaml
nameReference:
- kind: Deployment
fieldSpecs:
- path: spec/scaleTargetRef/name
kind: HorizontalPodAutoscaler
- kind: ReplicationController
fieldSpecs:
- path: spec/scaleTargetRef/name
kind: HorizontalPodAutoscaler
- kind: ReplicaSet
fieldSpecs:
- path: spec/scaleTargetRef/name
kind: HorizontalPodAutoscaler
- kind: StatefulSet
fieldSpecs:
- path: spec/scaleTargetRef/name
kind: HorizontalPodAutoscaler
- kind: ConfigMap
version: v1
fieldSpecs:
- path: spec/volumes/configMap/name
version: v1
kind: Pod
- path: spec/containers/env/valueFrom/configMapKeyRef/name
version: v1
kind: Pod
# ...
- path: spec/volumes/configMap/name
version: v1
kind: Pod
- path: spec/containers/env/valueFrom/configMapKeyRef/name
version: v1
kind: Pod
- path: spec/initContainers/env/valueFrom/configMapKeyRef/name
version: v1
kind: Pod
- path: spec/containers/envFrom/configMapRef/name
version: v1
kind: Pod
- path: spec/initContainers/envFrom/configMapRef/name
version: v1
kind: Pod
- path: spec/volumes/projected/sources/configMap/name
version: v1
kind: Pod
- path: template/spec/volumes/configMap/name
kind: PodTemplate
- path: spec/template/spec/volumes/configMap/name
kind: Deployment
- path: spec/template/spec/containers/env/valueFrom/configMapKeyRef/name
kind: Deployment
- path: spec/template/spec/initContainers/env/valueFrom/configMapKeyRef/name
kind: Deployment
- path: spec/template/spec/containers/envFrom/configMapRef/name
kind: Deployment
- path: spec/template/spec/initContainers/envFrom/configMapRef/name
kind: Deployment
- path: spec/template/spec/volumes/projected/sources/configMap/name
kind: Deployment
- path: spec/template/spec/volumes/configMap/name
kind: ReplicaSet
- path: spec/template/spec/containers/env/valueFrom/configMapKeyRef/name
kind: ReplicaSet
- path: spec/template/spec/initContainers/env/valueFrom/configMapKeyRef/name
kind: ReplicaSet
- path: spec/template/spec/containers/envFrom/configMapRef/name
kind: ReplicaSet
- path: spec/template/spec/initContainers/envFrom/configMapRef/name
kind: ReplicaSet
- path: spec/template/spec/volumes/projected/sources/configMap/name
kind: ReplicaSet
- path: spec/template/spec/volumes/configMap/name
kind: DaemonSet
- path: spec/template/spec/containers/env/valueFrom/configMapKeyRef/name
kind: DaemonSet
- path: spec/template/spec/initContainers/env/valueFrom/configMapKeyRef/name
kind: DaemonSet
- path: spec/template/spec/containers/envFrom/configMapRef/name
kind: DaemonSet
- path: spec/template/spec/initContainers/envFrom/configMapRef/name
kind: DaemonSet
- path: spec/template/spec/volumes/projected/sources/configMap/name
kind: DaemonSet
- path: spec/template/spec/volumes/configMap/name
kind: StatefulSet
- path: spec/template/spec/containers/env/valueFrom/configMapKeyRef/name
kind: StatefulSet
- path: spec/template/spec/initContainers/env/valueFrom/configMapKeyRef/name
kind: StatefulSet
- path: spec/template/spec/containers/envFrom/configMapRef/name
kind: StatefulSet
- path: spec/template/spec/initContainers/envFrom/configMapRef/name
kind: StatefulSet
- path: spec/template/spec/volumes/projected/sources/configMap/name
kind: StatefulSet
- path: spec/template/spec/volumes/configMap/name
kind: Job
- path: spec/template/spec/containers/env/valueFrom/configMapKeyRef/name
kind: Job
- path: spec/template/spec/initContainers/env/valueFrom/configMapKeyRef/name
kind: Job
- path: spec/template/spec/containers/envFrom/configMapRef/name
kind: Job
- path: spec/template/spec/initContainers/envFrom/configMapRef/name
kind: Job
- path: spec/template/spec/volumes/projected/sources/configMap/name
kind: Job
- path: spec/jobTemplate/spec/template/spec/volumes/configMap/name
kind: CronJob
- path: spec/jobTemplate/spec/template/spec/volumes/projected/sources/configMap/name
kind: CronJob
- path: spec/jobTemplate/spec/template/spec/containers/env/valueFrom/configMapKeyRef/name
kind: CronJob
- path: spec/jobTemplate/spec/template/spec/initContainers/env/valueFrom/configMapKeyRef/name
kind: CronJob
- path: spec/jobTemplate/spec/template/spec/containers/envFrom/configMapRef/name
kind: CronJob
- path: spec/jobTemplate/spec/template/spec/initContainers/envFrom/configMapRef/name
kind: CronJob
- path: spec/configSource/configMap
kind: Node
- path: rules/resourceNames
kind: Role
- path: rules/resourceNames
kind: ClusterRole
- path: metadata/annotations/nginx.ingress.kubernetes.io\/fastcgi-params-configmap
kind: Ingress
- kind: Secret
version: v1
fieldSpecs:
- path: spec/volumes/secret/secretName
version: v1
kind: Pod
- path: spec/containers/env/valueFrom/secretKeyRef/name
version: v1
kind: Pod
- path: spec/volumes/secret/secretName
version: v1
kind: Pod
- path: spec/containers/env/valueFrom/secretKeyRef/name
version: v1
kind: Pod
- path: spec/initContainers/env/valueFrom/secretKeyRef/name
version: v1
kind: Pod
- path: spec/containers/envFrom/secretRef/name
version: v1
kind: Pod
- path: spec/initContainers/envFrom/secretRef/name
version: v1
kind: Pod
- path: spec/imagePullSecrets/name
version: v1
kind: Pod
- path: spec/volumes/projected/sources/secret/name
version: v1
kind: Pod
- path: spec/template/spec/volumes/secret/secretName
kind: Deployment
- path: spec/template/spec/containers/env/valueFrom/secretKeyRef/name
kind: Deployment
- path: spec/template/spec/initContainers/env/valueFrom/secretKeyRef/name
kind: Deployment
- path: spec/template/spec/containers/envFrom/secretRef/name
kind: Deployment
- path: spec/template/spec/initContainers/envFrom/secretRef/name
kind: Deployment
- path: spec/template/spec/imagePullSecrets/name
kind: Deployment
- path: spec/template/spec/volumes/projected/sources/secret/name
kind: Deployment
- path: spec/template/spec/volumes/secret/secretName
kind: ReplicaSet
- path: spec/template/spec/containers/env/valueFrom/secretKeyRef/name
kind: ReplicaSet
- path: spec/template/spec/initContainers/env/valueFrom/secretKeyRef/name
kind: ReplicaSet
- path: spec/template/spec/containers/envFrom/secretRef/name
kind: ReplicaSet
- path: spec/template/spec/initContainers/envFrom/secretRef/name
kind: ReplicaSet
- path: spec/template/spec/imagePullSecrets/name
kind: ReplicaSet
- path: spec/template/spec/volumes/projected/sources/secret/name
kind: ReplicaSet
- path: spec/template/spec/volumes/secret/secretName
kind: DaemonSet
- path: spec/template/spec/containers/env/valueFrom/secretKeyRef/name
kind: DaemonSet
- path: spec/template/spec/initContainers/env/valueFrom/secretKeyRef/name
kind: DaemonSet
- path: spec/template/spec/containers/envFrom/secretRef/name
kind: DaemonSet
- path: spec/template/spec/initContainers/envFrom/secretRef/name
kind: DaemonSet
- path: spec/template/spec/imagePullSecrets/name
kind: DaemonSet
- path: spec/template/spec/volumes/projected/sources/secret/name
kind: DaemonSet
- path: spec/template/spec/volumes/secret/secretName
kind: StatefulSet
- path: spec/template/spec/containers/env/valueFrom/secretKeyRef/name
kind: StatefulSet
- path: spec/template/spec/initContainers/env/valueFrom/secretKeyRef/name
kind: StatefulSet
- path: spec/template/spec/containers/envFrom/secretRef/name
kind: StatefulSet
- path: spec/template/spec/initContainers/envFrom/secretRef/name
kind: StatefulSet
- path: spec/template/spec/imagePullSecrets/name
kind: StatefulSet
- path: spec/template/spec/volumes/projected/sources/secret/name
kind: StatefulSet
- path: spec/template/spec/volumes/secret/secretName
kind: Job
- path: spec/template/spec/containers/env/valueFrom/secretKeyRef/name
kind: Job
- path: spec/template/spec/initContainers/env/valueFrom/secretKeyRef/name
kind: Job
- path: spec/template/spec/containers/envFrom/secretRef/name
kind: Job
- path: spec/template/spec/initContainers/envFrom/secretRef/name
kind: Job
- path: spec/template/spec/imagePullSecrets/name
kind: Job
- path: spec/template/spec/volumes/projected/sources/secret/name
kind: Job
- path: spec/jobTemplate/spec/template/spec/volumes/secret/secretName
kind: CronJob
- path: spec/jobTemplate/spec/template/spec/volumes/projected/sources/secret/name
kind: CronJob
- path: spec/jobTemplate/spec/template/spec/containers/env/valueFrom/secretKeyRef/name
kind: CronJob
- path: spec/jobTemplate/spec/template/spec/initContainers/env/valueFrom/secretKeyRef/name
kind: CronJob
- path: spec/jobTemplate/spec/template/spec/containers/envFrom/secretRef/name
kind: CronJob
- path: spec/jobTemplate/spec/template/spec/initContainers/envFrom/secretRef/name
kind: CronJob
- path: spec/jobTemplate/spec/template/spec/imagePullSecrets/name
kind: CronJob
- path: spec/tls/secretName
kind: Ingress
- path: metadata/annotations/ingress.kubernetes.io\/auth-secret
kind: Ingress
- path: metadata/annotations/nginx.ingress.kubernetes.io\/auth-secret
kind: Ingress
- path: metadata/annotations/nginx.ingress.kubernetes.io\/auth-tls-secret
kind: Ingress
- path: spec/tls/secretName
kind: Ingress
- path: imagePullSecrets/name
kind: ServiceAccount
- path: parameters/secretName
kind: StorageClass
- path: parameters/adminSecretName
kind: StorageClass
- path: parameters/userSecretName
kind: StorageClass
- path: parameters/secretRef
kind: StorageClass
- path: rules/resourceNames
kind: Role
- path: rules/resourceNames
kind: ClusterRole
- path: spec/template/spec/containers/env/valueFrom/secretKeyRef/name
kind: Service
group: serving.knative.dev
version: v1
- path: spec/azureFile/secretName
kind: PersistentVolume
- kind: Service
version: v1
fieldSpecs:
- path: spec/serviceName
kind: StatefulSet
group: apps
- path: spec/rules/http/paths/backend/serviceName
kind: Ingress
- path: spec/backend/serviceName
kind: Ingress
- path: spec/rules/http/paths/backend/service/name
kind: Ingress
- path: spec/defaultBackend/service/name
kind: Ingress
- path: spec/service/name
kind: APIService
group: apiregistration.k8s.io
- path: webhooks/clientConfig/service
kind: ValidatingWebhookConfiguration
group: admissionregistration.k8s.io
- path: webhooks/clientConfig/service
kind: MutatingWebhookConfiguration
group: admissionregistration.k8s.io
- kind: Role
group: rbac.authorization.k8s.io
fieldSpecs:
- path: roleRef/name
kind: RoleBinding
group: rbac.authorization.k8s.io
- kind: ClusterRole
group: rbac.authorization.k8s.io
fieldSpecs:
- path: roleRef/name
kind: RoleBinding
group: rbac.authorization.k8s.io
- path: roleRef/name
kind: ClusterRoleBinding
group: rbac.authorization.k8s.io
- kind: ServiceAccount
version: v1
fieldSpecs:
- path: subjects
kind: RoleBinding
group: rbac.authorization.k8s.io
- path: subjects
kind: ClusterRoleBinding
group: rbac.authorization.k8s.io
- path: spec/serviceAccountName
kind: Pod
- path: spec/template/spec/serviceAccountName
kind: StatefulSet
- path: spec/template/spec/serviceAccountName
kind: Deployment
- path: spec/template/spec/serviceAccountName
kind: ReplicationController
- path: spec/jobTemplate/spec/template/spec/serviceAccountName
kind: CronJob
- path: spec/template/spec/serviceAccountName
kind: Job
- path: spec/template/spec/serviceAccountName
kind: DaemonSet
- kind: PersistentVolumeClaim
version: v1
fieldSpecs:
- path: spec/volumes/persistentVolumeClaim/claimName
kind: Pod
- path: spec/template/spec/volumes/persistentVolumeClaim/claimName
kind: StatefulSet
- path: spec/template/spec/volumes/persistentVolumeClaim/claimName
kind: Deployment
- path: spec/template/spec/volumes/persistentVolumeClaim/claimName
kind: ReplicationController
- path: spec/jobTemplate/spec/template/spec/volumes/persistentVolumeClaim/claimName
kind: CronJob
- path: spec/template/spec/volumes/persistentVolumeClaim/claimName
kind: Job
- path: spec/template/spec/volumes/persistentVolumeClaim/claimName
kind: DaemonSet
- kind: PersistentVolume
version: v1
fieldSpecs:
- path: spec/volumeName
kind: PersistentVolumeClaim
- path: rules/resourceNames
kind: ClusterRole
- kind: StorageClass
version: v1
group: storage.k8s.io
fieldSpecs:
- path: spec/storageClassName
kind: PersistentVolume
- path: spec/storageClassName
kind: PersistentVolumeClaim
- path: spec/volumeClaimTemplates/spec/storageClassName
kind: StatefulSet
- kind: PriorityClass
version: v1
group: scheduling.k8s.io
fieldSpecs:
- path: spec/priorityClassName
kind: Pod
- path: spec/template/spec/priorityClassName
kind: StatefulSet
- path: spec/template/spec/priorityClassName
kind: Deployment
- path: spec/template/spec/priorityClassName
kind: ReplicationController
- path: spec/jobTemplate/spec/template/spec/priorityClassName
kind: CronJob
- path: spec/template/spec/priorityClassName
kind: Job
- path: spec/template/spec/priorityClassName
kind: DaemonSet
- kind: IngressClass
version: v1
group: networking.k8s.io/v1
fieldSpecs:
- path: spec/ingressClassName
kind: Ingress
```
## Customizing transformer configurations

View File

@@ -1,23 +0,0 @@
#!/usr/bin/env bash
set -e
if [[ "$PULL_NUMBER" -ne "" ]]; then
cmd="$MYGOBIN/prchecker
-owner=$REPO_OWNER
-repo=$REPO_NAME
-pr=$PULL_NUMBER
$MODULES"
echo $MYGOBIN
echo $REPO_OWNER
echo $REPO_NAME
echo $PULL_NUMBER
echo $MODULES
eval $cmd
else
echo "Multi-module check skipped. No PULL_NUMBER provided.
To run this check locally set PULL_NUMBER to the PR ID from GitHub."
fi

View File

@@ -11,6 +11,8 @@ builtinPlugins=(AnnotationsTransformer \
PatchStrategicMergeTransformer \
PatchTransformer \
PrefixSuffixTransformer \
PrefixTransformer \
SuffixTransformer \
ReplicaCountTransformer \
SecretGenerator \
ValueAddTransformer \

View File

@@ -8,7 +8,7 @@ require (
github.com/spf13/cobra v1.0.0
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.7.0
sigs.k8s.io/kustomize/api v0.10.0
sigs.k8s.io/kustomize/api v0.10.1
sigs.k8s.io/kustomize/cmd/config v0.10.2
sigs.k8s.io/kustomize/kyaml v0.13.0
sigs.k8s.io/yaml v1.2.0
@@ -20,3 +20,7 @@ exclude (
)
replace sigs.k8s.io/kustomize/api => ../api
replace sigs.k8s.io/kustomize/cmd/config => ../cmd/config
replace sigs.k8s.io/kustomize/kyaml => ../kyaml

View File

@@ -253,10 +253,6 @@ k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8
k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE=
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e h1:KLHHjkdQFomZy8+06csTWZ0m1343QqxZhR2LJ1OxCYM=
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e/go.mod h1:vHXdDvt9+2spS2Rx9ql3I8tycm3H9FDfdUoIuKCefvw=
sigs.k8s.io/kustomize/cmd/config v0.10.2 h1:2GD3+knDaqZo6rSibkc4kKGp8auNBJrGPZQCTWN4Rtc=
sigs.k8s.io/kustomize/cmd/config v0.10.2/go.mod h1:K2aW7nXJ0AaT+VA/eO0/dzFLxmpFcTzudmAgDwPY1HQ=
sigs.k8s.io/kustomize/kyaml v0.13.0 h1:9c+ETyNfSrVhxvphs+K2dzT3dh5oVPPEqPOE/cUpScY=
sigs.k8s.io/kustomize/kyaml v0.13.0/go.mod h1:FTJxEZ86ScK184NpGSAQcfEqee0nul8oLCK30D47m4E=
sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw=
sigs.k8s.io/yaml v1.2.0 h1:kr/MCeFWJWTwyaHoR9c8EjH9OumOmoF9YGiZd7lFm/Q=
sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc=

View File

@@ -4,6 +4,7 @@
package framework
import (
goerrors "errors"
"os"
"sigs.k8s.io/kustomize/kyaml/errors"
@@ -116,8 +117,21 @@ func Execute(p ResourceListProcessor, rlSource *kio.ByteReadWriter) error {
}
rl.FunctionConfig = rlSource.FunctionConfig
// We store the original
nodeAnnos, err := kio.PreprocessResourcesForInternalAnnotationMigration(rl.Items)
if err != nil {
return err
}
retErr := p.Process(&rl)
// If either the internal annotations for path, index, and id OR the legacy
// annotations for path, index, and id are changed, we have to update the other.
err = kio.ReconcileInternalAnnotations(rl.Items, nodeAnnos)
if err != nil {
return err
}
// Write the results
// Set the ResourceList.results for validating functions
if len(rl.Results) > 0 {
@@ -140,8 +154,19 @@ func Execute(p ResourceListProcessor, rlSource *kio.ByteReadWriter) error {
// Filter executes the given kio.Filter and replaces the ResourceList's items with the result.
// This can be used to help implement ResourceListProcessors. See SimpleProcessor for example.
//
// Filters that return a Result as error will store the result in the ResourceList
// and continue processing instead of erroring out.
func (rl *ResourceList) Filter(api kio.Filter) error {
var err error
rl.Items, err = api.Filter(rl.Items)
return errors.Wrap(err)
if err != nil {
var r Results
if goerrors.As(err, &r) {
rl.Results = append(rl.Results, r...)
return nil
}
return errors.Wrap(err)
}
return nil
}

View File

@@ -15,8 +15,8 @@ import (
)
func TestExecute_Result(t *testing.T) {
p := framework.ResourceListProcessorFunc(func(rl *framework.ResourceList) error {
err := &framework.Results{
p := framework.SimpleProcessor{Filter: kio.FilterFunc(func(in []*yaml.RNode) ([]*yaml.RNode, error) {
return in, framework.Results{
{
Message: "bad value for replicas",
Severity: framework.Error,
@@ -40,9 +40,7 @@ func TestExecute_Result(t *testing.T) {
Tags: map[string]string{"foo": "bar"},
},
}
rl.Results = *err
return err
})
})}
out := new(bytes.Buffer)
source := &kio.ByteReadWriter{Reader: bytes.NewBufferString(`
kind: ResourceList
@@ -57,10 +55,7 @@ items:
replicas: 0
`), Writer: out}
err := framework.Execute(p, source)
assert.EqualError(t, err, `[error] v1/Deployment/default/tester .spec.Replicas: bad value for replicas
[error]: some error`)
assert.Equal(t, 1, err.(*framework.Results).ExitCode())
assert.NoError(t, err)
assert.Equal(t, `apiVersion: config.kubernetes.io/v1
kind: ResourceList
items:

View File

@@ -5,6 +5,7 @@ package framework
import (
"fmt"
"sort"
"strings"
"sigs.k8s.io/kustomize/kyaml/yaml"
@@ -64,7 +65,12 @@ func (i Result) String() string {
}
}
formatString := "[%s]"
list := []interface{}{i.Severity}
severity := i.Severity
// We default Severity to Info when converting a result to a message.
if i.Severity == "" {
severity = Info
}
list := []interface{}{severity}
if len(idStringList) > 0 {
formatString += " %s"
list = append(list, strings.Join(idStringList, "/"))
@@ -121,3 +127,61 @@ func (e Results) ExitCode() int {
}
return 0
}
// Sort performs an in place stable sort of Results
func (e Results) Sort() {
sort.SliceStable(e, func(i, j int) bool {
if fileLess(e, i, j) != 0 {
return fileLess(e, i, j) < 0
}
if severityLess(e, i, j) != 0 {
return severityLess(e, i, j) < 0
}
return resultToString(*e[i]) < resultToString(*e[j])
})
}
func severityLess(items Results, i, j int) int {
severityToNumber := map[Severity]int{
Error: 0,
Warning: 1,
Info: 2,
}
severityLevelI, found := severityToNumber[items[i].Severity]
if !found {
severityLevelI = 3
}
severityLevelJ, found := severityToNumber[items[j].Severity]
if !found {
severityLevelJ = 3
}
return severityLevelI - severityLevelJ
}
func fileLess(items Results, i, j int) int {
var fileI, fileJ File
if items[i].File == nil {
fileI = File{}
} else {
fileI = *items[i].File
}
if items[j].File == nil {
fileJ = File{}
} else {
fileJ = *items[j].File
}
if fileI.Path != fileJ.Path {
if fileI.Path < fileJ.Path {
return -1
} else {
return 1
}
}
return fileI.Index - fileJ.Index
}
func resultToString(item Result) string {
return fmt.Sprintf("resource-ref:%s,field:%s,message:%s",
item.ResourceRef, item.Field, item.Message)
}

View File

@@ -0,0 +1,274 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package framework_test
import (
"reflect"
"testing"
"sigs.k8s.io/kustomize/kyaml/fn/framework"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
func TestResults_Sort(t *testing.T) {
testcases := []struct {
name string
input framework.Results
output framework.Results
}{
{
name: "sort based on severity",
input: framework.Results{
{
Message: "Error message 1",
Severity: framework.Info,
},
{
Message: "Error message 2",
Severity: framework.Error,
},
},
output: framework.Results{
{
Message: "Error message 2",
Severity: framework.Error,
},
{
Message: "Error message 1",
Severity: framework.Info,
},
},
},
{
name: "sort based on file",
input: framework.Results{
{
Message: "Error message",
Severity: framework.Error,
File: &framework.File{
Path: "resource.yaml",
Index: 1,
},
},
{
Message: "Error message",
Severity: framework.Info,
File: &framework.File{
Path: "resource.yaml",
Index: 0,
},
},
{
Message: "Error message",
Severity: framework.Info,
File: &framework.File{
Path: "other-resource.yaml",
Index: 0,
},
},
{
Message: "Error message",
Severity: framework.Warning,
File: &framework.File{
Path: "resource.yaml",
Index: 2,
},
},
{
Message: "Error message",
Severity: framework.Warning,
},
},
output: framework.Results{
{
Message: "Error message",
Severity: framework.Warning,
},
{
Message: "Error message",
Severity: framework.Info,
File: &framework.File{
Path: "other-resource.yaml",
Index: 0,
},
},
{
Message: "Error message",
Severity: framework.Info,
File: &framework.File{
Path: "resource.yaml",
Index: 0,
},
},
{
Message: "Error message",
Severity: framework.Error,
File: &framework.File{
Path: "resource.yaml",
Index: 1,
},
},
{
Message: "Error message",
Severity: framework.Warning,
File: &framework.File{
Path: "resource.yaml",
Index: 2,
},
},
},
},
{
name: "sort based on other fields",
input: framework.Results{
{
Message: "Error message",
Severity: framework.Error,
ResourceRef: &yaml.ResourceIdentifier{
TypeMeta: yaml.TypeMeta{
APIVersion: "v1",
Kind: "Pod",
},
NameMeta: yaml.NameMeta{
Namespace: "foo-ns",
Name: "bar",
},
},
Field: &framework.Field{
Path: "spec",
},
},
{
Message: "Error message",
Severity: framework.Error,
ResourceRef: &yaml.ResourceIdentifier{
TypeMeta: yaml.TypeMeta{
APIVersion: "v1",
Kind: "Pod",
},
NameMeta: yaml.NameMeta{
Namespace: "foo-ns",
Name: "bar",
},
},
Field: &framework.Field{
Path: "metadata.name",
},
},
{
Message: "Another error message",
Severity: framework.Error,
ResourceRef: &yaml.ResourceIdentifier{
TypeMeta: yaml.TypeMeta{
APIVersion: "v1",
Kind: "Pod",
},
NameMeta: yaml.NameMeta{
Namespace: "foo-ns",
Name: "bar",
},
},
Field: &framework.Field{
Path: "metadata.name",
},
},
{
Message: "Another error message",
Severity: framework.Error,
ResourceRef: &yaml.ResourceIdentifier{
TypeMeta: yaml.TypeMeta{
APIVersion: "v1",
Kind: "ConfigMap",
},
NameMeta: yaml.NameMeta{
Namespace: "foo-ns",
Name: "bar",
},
},
Field: &framework.Field{
Path: "metadata.name",
},
},
},
output: framework.Results{
{
Message: "Another error message",
Severity: framework.Error,
ResourceRef: &yaml.ResourceIdentifier{
TypeMeta: yaml.TypeMeta{
APIVersion: "v1",
Kind: "ConfigMap",
},
NameMeta: yaml.NameMeta{
Namespace: "foo-ns",
Name: "bar",
},
},
Field: &framework.Field{
Path: "metadata.name",
},
},
{
Message: "Another error message",
Severity: framework.Error,
ResourceRef: &yaml.ResourceIdentifier{
TypeMeta: yaml.TypeMeta{
APIVersion: "v1",
Kind: "Pod",
},
NameMeta: yaml.NameMeta{
Namespace: "foo-ns",
Name: "bar",
},
},
Field: &framework.Field{
Path: "metadata.name",
},
},
{
Message: "Error message",
Severity: framework.Error,
ResourceRef: &yaml.ResourceIdentifier{
TypeMeta: yaml.TypeMeta{
APIVersion: "v1",
Kind: "Pod",
},
NameMeta: yaml.NameMeta{
Namespace: "foo-ns",
Name: "bar",
},
},
Field: &framework.Field{
Path: "metadata.name",
},
},
{
Message: "Error message",
Severity: framework.Error,
ResourceRef: &yaml.ResourceIdentifier{
TypeMeta: yaml.TypeMeta{
APIVersion: "v1",
Kind: "Pod",
},
NameMeta: yaml.NameMeta{
Namespace: "foo-ns",
Name: "bar",
},
},
Field: &framework.Field{
Path: "spec",
},
},
},
},
}
for _, tc := range testcases {
tc.input.Sort()
if !reflect.DeepEqual(tc.input, tc.output) {
t.Errorf("in testcase %q, expect: %#v, but got: %#v", tc.name, tc.output, tc.input)
}
}
}

View File

@@ -400,7 +400,6 @@ metadata:
"a": "a long string that would certainly see a newline introduced by the YAML marshaller abcd123",
"metadata": {
"annotations": {
"config.kubernetes.io/path": "test.json",
"internal.config.kubernetes.io/path": "test.json"
}
}
@@ -429,7 +428,6 @@ metadata:
"a": "a long string that would certainly see a newline introduced by the YAML marshaller abcd123",
"metadata": {
"annotations": {
"config.kubernetes.io/path": "test.json",
"internal.config.kubernetes.io/path": "test.json"
}
}
@@ -449,7 +447,6 @@ metadata:
"a": "b",
"metadata": {
"annotations": {
"config.kubernetes.io/path": "test.json",
"internal.config.kubernetes.io/path": "test.json"
}
}

View File

@@ -7,6 +7,7 @@ package kio
import (
"fmt"
"strconv"
"sigs.k8s.io/kustomize/kyaml/errors"
"sigs.k8s.io/kustomize/kyaml/kio/kioutil"
@@ -114,13 +115,11 @@ func (p Pipeline) ExecuteWithCallback(callback PipelineExecuteCallbackFunc) erro
}
// apply operations
var err error
for i := range p.Filters {
// Not all RNodes passed through kio.Pipeline have metadata nor should
// they all be required to.
var nodeAnnos map[string]map[string]string
nodeAnnos, err = storeInternalAnnotations(result)
if err != nil && err != yaml.ErrMissingMetadata {
nodeAnnos, err := PreprocessResourcesForInternalAnnotationMigration(result)
if err != nil {
return err
}
@@ -138,8 +137,8 @@ func (p Pipeline) ExecuteWithCallback(callback PipelineExecuteCallbackFunc) erro
// If either the internal annotations for path, index, and id OR the legacy
// annotations for path, index, and id are changed, we have to update the other.
err = reconcileInternalAnnotations(result, nodeAnnos)
if err != nil && err != yaml.ErrMissingMetadata {
err = ReconcileInternalAnnotations(result, nodeAnnos)
if err != nil {
return err
}
}
@@ -166,55 +165,49 @@ func FilterAll(filter yaml.Filter) Filter {
})
}
// Store the original path, index, and id annotations so that we can reconcile
// it later. This is necessary because currently both internal-prefixed annotations
// PreprocessResourcesForInternalAnnotationMigration returns a mapping from id to all
// internal annotations, so that we can use it to reconcile the annotations
// later. This is necessary because currently both internal-prefixed annotations
// and legacy annotations are currently supported, and a change to one must be
// reflected in the other.
func storeInternalAnnotations(result []*yaml.RNode) (map[string]map[string]string, error) {
nodeAnnosMap := make(map[string]map[string]string)
// reflected in the other if needed.
func PreprocessResourcesForInternalAnnotationMigration(result []*yaml.RNode) (map[string]map[string]string, error) {
idToAnnosMap := make(map[string]map[string]string)
for i := range result {
if err := kioutil.CopyLegacyAnnotations(result[i]); err != nil {
return nil, err
}
meta, err := result[i].GetMeta()
idStr := strconv.Itoa(i)
err := result[i].PipeE(yaml.SetAnnotation(kioutil.InternalAnnotationsMigrationResourceIDAnnotation, idStr))
if err != nil {
return nil, err
}
if err := checkMismatchedAnnos(meta); err != nil {
idToAnnosMap[idStr] = kioutil.GetInternalAnnotations(result[i])
if err = kioutil.CopyLegacyAnnotations(result[i]); err != nil {
return nil, err
}
path := meta.Annotations[kioutil.PathAnnotation]
index := meta.Annotations[kioutil.IndexAnnotation]
id := meta.Annotations[kioutil.IdAnnotation]
if _, ok := nodeAnnosMap[path]; !ok {
nodeAnnosMap[path] = make(map[string]string)
meta, _ := result[i].GetMeta()
if err = checkMismatchedAnnos(meta.Annotations); err != nil {
return nil, err
}
nodeAnnosMap[path][index] = id
}
return nodeAnnosMap, nil
return idToAnnosMap, nil
}
func checkMismatchedAnnos(meta yaml.ResourceMeta) error {
path := meta.Annotations[kioutil.PathAnnotation]
index := meta.Annotations[kioutil.IndexAnnotation]
id := meta.Annotations[kioutil.IdAnnotation]
func checkMismatchedAnnos(annotations map[string]string) error {
path := annotations[kioutil.PathAnnotation]
index := annotations[kioutil.IndexAnnotation]
id := annotations[kioutil.IdAnnotation]
legacyPath := meta.Annotations[kioutil.LegacyPathAnnotation]
legacyIndex := meta.Annotations[kioutil.LegacyIndexAnnotation]
legacyId := meta.Annotations[kioutil.LegacyIdAnnotation]
legacyPath := annotations[kioutil.LegacyPathAnnotation]
legacyIndex := annotations[kioutil.LegacyIndexAnnotation]
legacyId := annotations[kioutil.LegacyIdAnnotation]
// if prior to running the functions, the legacy and internal annotations differ,
// throw an error as we cannot infer the user's intent.
if path != legacyPath {
if path != "" && legacyPath != "" && path != legacyPath {
return fmt.Errorf("resource input to function has mismatched legacy and internal path annotations")
}
if index != legacyIndex {
if index != "" && legacyIndex != "" && index != legacyIndex {
return fmt.Errorf("resource input to function has mismatched legacy and internal index annotations")
}
if id != legacyId {
if id != "" && legacyId != "" && id != legacyId {
return fmt.Errorf("resource input to function has mismatched legacy and internal id annotations")
}
return nil
@@ -226,53 +219,115 @@ type nodeAnnotations struct {
id string
}
func reconcileInternalAnnotations(result []*yaml.RNode, nodeAnnosMap map[string]map[string]string) error {
for _, node := range result {
meta, err := node.GetMeta()
if err != nil {
return err
}
// ReconcileInternalAnnotations reconciles the annotation format for path, index and id annotations.
// It will ensure the output annotation format matches the format in the input. e.g. if the input
// format uses the legacy format and the output will be converted to the legacy format if it's not.
func ReconcileInternalAnnotations(result []*yaml.RNode, nodeAnnosMap map[string]map[string]string) error {
useInternal, useLegacy, err := determineAnnotationsFormat(nodeAnnosMap)
if err != nil {
return err
}
for i := range result {
// if only one annotation is set, set the other.
err = missingInternalOrLegacyAnnotations(node, meta)
err = missingInternalOrLegacyAnnotations(result[i])
if err != nil {
return err
}
// we must check to see if the function changed either the new internal annotations
// or the old legacy annotations. If one is changed, the change must be reflected
// in the other.
err = checkAnnotationsAltered(node, meta, nodeAnnosMap)
err = checkAnnotationsAltered(result[i], nodeAnnosMap)
if err != nil {
return err
}
// We invoke determineAnnotationsFormat to find out if the original annotations
// use the internal or (and) the legacy format. We format the resources to
// make them consistent with original format.
err = formatInternalAnnotations(result[i], useInternal, useLegacy)
if err != nil {
return err
}
// if the annotations are still somehow out of sync, throw an error
meta, err = node.GetMeta()
meta, _ := result[i].GetMeta()
err = checkMismatchedAnnos(meta.Annotations)
if err != nil {
return err
}
err = checkMismatchedAnnos(meta)
if err != nil {
if _, err = result[i].Pipe(yaml.ClearAnnotation(kioutil.InternalAnnotationsMigrationResourceIDAnnotation)); err != nil {
return err
}
}
return nil
}
func missingInternalOrLegacyAnnotations(rn *yaml.RNode, meta yaml.ResourceMeta) error {
if err := missingInternalOrLegacyAnnotation(rn, meta, kioutil.PathAnnotation, kioutil.LegacyPathAnnotation); err != nil {
// determineAnnotationsFormat determines if the resources are using one of the internal and legacy annotation format or both of them.
func determineAnnotationsFormat(nodeAnnosMap map[string]map[string]string) (useInternal, useLegacy bool, err error) {
if len(nodeAnnosMap) == 0 {
return true, true, nil
}
var internal, legacy *bool
for _, annos := range nodeAnnosMap {
_, foundPath := annos[kioutil.PathAnnotation]
_, foundIndex := annos[kioutil.IndexAnnotation]
_, foundId := annos[kioutil.IdAnnotation]
_, foundLegacyPath := annos[kioutil.LegacyPathAnnotation]
_, foundLegacyIndex := annos[kioutil.LegacyIndexAnnotation]
_, foundLegacyId := annos[kioutil.LegacyIdAnnotation]
if !(foundPath || foundIndex || foundId || foundLegacyPath || foundLegacyIndex || foundLegacyId) {
continue
}
foundOneOf := foundPath || foundIndex || foundId
if internal == nil {
f := foundOneOf
internal = &f
}
if (foundOneOf && !*internal) || (!foundOneOf && *internal) {
err = fmt.Errorf("the annotation formatting in the input resources is not consistent")
return
}
foundOneOf = foundLegacyPath || foundLegacyIndex || foundLegacyId
if legacy == nil {
f := foundOneOf
legacy = &f
}
if (foundOneOf && !*legacy) || (!foundOneOf && *legacy) {
err = fmt.Errorf("the annotation formatting in the input resources is not consistent")
return
}
}
if internal != nil {
useInternal = *internal
}
if legacy != nil {
useLegacy = *legacy
}
return
}
func missingInternalOrLegacyAnnotations(rn *yaml.RNode) error {
if err := missingInternalOrLegacyAnnotation(rn, kioutil.PathAnnotation, kioutil.LegacyPathAnnotation); err != nil {
return err
}
if err := missingInternalOrLegacyAnnotation(rn, meta, kioutil.IndexAnnotation, kioutil.LegacyIndexAnnotation); err != nil {
if err := missingInternalOrLegacyAnnotation(rn, kioutil.IndexAnnotation, kioutil.LegacyIndexAnnotation); err != nil {
return err
}
if err := missingInternalOrLegacyAnnotation(rn, meta, kioutil.IdAnnotation, kioutil.LegacyIdAnnotation); err != nil {
if err := missingInternalOrLegacyAnnotation(rn, kioutil.IdAnnotation, kioutil.LegacyIdAnnotation); err != nil {
return err
}
return nil
}
func missingInternalOrLegacyAnnotation(rn *yaml.RNode, meta yaml.ResourceMeta, newKey string, legacyKey string) error {
value := meta.Annotations[newKey]
legacyValue := meta.Annotations[legacyKey]
func missingInternalOrLegacyAnnotation(rn *yaml.RNode, newKey string, legacyKey string) error {
meta, _ := rn.GetMeta()
annotations := meta.Annotations
value := annotations[newKey]
legacyValue := annotations[legacyKey]
if value == "" && legacyValue == "" {
// do nothing
@@ -293,98 +348,88 @@ func missingInternalOrLegacyAnnotation(rn *yaml.RNode, meta yaml.ResourceMeta, n
return nil
}
func checkAnnotationsAltered(rn *yaml.RNode, meta yaml.ResourceMeta, nodeAnnosMap map[string]map[string]string) error {
func checkAnnotationsAltered(rn *yaml.RNode, nodeAnnosMap map[string]map[string]string) error {
meta, _ := rn.GetMeta()
annotations := meta.Annotations
// get the resource's current path, index, and ids from the new annotations
internal := nodeAnnotations{
path: meta.Annotations[kioutil.PathAnnotation],
index: meta.Annotations[kioutil.IndexAnnotation],
id: meta.Annotations[kioutil.IdAnnotation],
path: annotations[kioutil.PathAnnotation],
index: annotations[kioutil.IndexAnnotation],
id: annotations[kioutil.IdAnnotation],
}
// get the resource's current path, index, and ids from the legacy annotations
legacy := nodeAnnotations{
path: meta.Annotations[kioutil.LegacyPathAnnotation],
index: meta.Annotations[kioutil.LegacyIndexAnnotation],
id: meta.Annotations[kioutil.LegacyIdAnnotation],
path: annotations[kioutil.LegacyPathAnnotation],
index: annotations[kioutil.LegacyIndexAnnotation],
id: annotations[kioutil.LegacyIdAnnotation],
}
if internal.path == legacy.path &&
internal.index == legacy.index &&
internal.id == legacy.id {
// none of the annotations differ, so no reconciliation is needed
rid := annotations[kioutil.InternalAnnotationsMigrationResourceIDAnnotation]
originalAnnotations, found := nodeAnnosMap[rid]
if !found {
return nil
}
// nodeAnnosMap is a map of structure path -> index -> id that stores
// all of the resources' path/index/id annotations prior to the functions
// being run. We use that to check whether the legacy or new internal
// annotations have been changed, and make sure the change is reflected
// in the other.
// first, check if the internal annotations are found in nodeAnnosMap
if indexIdMap, ok := nodeAnnosMap[internal.path]; ok {
if id, ok := indexIdMap[internal.index]; ok {
if id == internal.id {
// the internal annotations of the resource match the ones stored in
// nodeAnnosMap, so we should copy the legacy annotations to the
// internal ones
if err := updateAnnotations(rn, meta,
[]string{
kioutil.PathAnnotation,
kioutil.IndexAnnotation,
kioutil.IdAnnotation,
},
[]string{
legacy.path,
legacy.index,
legacy.id,
}); err != nil {
return err
}
originalPath, found := originalAnnotations[kioutil.PathAnnotation]
if !found {
originalPath = originalAnnotations[kioutil.LegacyPathAnnotation]
}
if originalPath != "" {
if originalPath != internal.path && originalPath != legacy.path && internal.path != legacy.path {
return fmt.Errorf("resource input to function has mismatched legacy and internal path annotations")
} else if originalPath != internal.path {
if _, err := rn.Pipe(yaml.SetAnnotation(kioutil.LegacyPathAnnotation, internal.path)); err != nil {
return err
}
} else if originalPath != legacy.path {
if _, err := rn.Pipe(yaml.SetAnnotation(kioutil.PathAnnotation, legacy.path)); err != nil {
return err
}
}
}
// check the opposite, to see if the legacy annotations are in nodeAnnosMap
if indexIdMap, ok := nodeAnnosMap[legacy.path]; ok {
if id, ok := indexIdMap[legacy.index]; ok {
if id == legacy.id {
// the legacy annotations of the resource match the ones stored in
// nodeAnnosMap, so we should copy the internal annotations to the
// legacy ones
if err := updateAnnotations(rn, meta,
[]string{
kioutil.LegacyPathAnnotation,
kioutil.LegacyIndexAnnotation,
kioutil.LegacyIdAnnotation,
},
[]string{
internal.path,
internal.index,
internal.id,
}); err != nil {
return err
}
originalIndex, found := originalAnnotations[kioutil.IndexAnnotation]
if !found {
originalIndex = originalAnnotations[kioutil.LegacyIndexAnnotation]
}
if originalIndex != "" {
if originalIndex != internal.index && originalIndex != legacy.index && internal.index != legacy.index {
return fmt.Errorf("resource input to function has mismatched legacy and internal index annotations")
} else if originalIndex != internal.index {
if _, err := rn.Pipe(yaml.SetAnnotation(kioutil.LegacyIndexAnnotation, internal.index)); err != nil {
return err
}
} else if originalIndex != legacy.index {
if _, err := rn.Pipe(yaml.SetAnnotation(kioutil.IndexAnnotation, legacy.index)); err != nil {
return err
}
}
}
return nil
}
func updateAnnotations(rn *yaml.RNode, meta yaml.ResourceMeta, keys []string, values []string) error {
if len(keys) != len(values) {
return fmt.Errorf("keys is not same length as values")
}
for i := range keys {
_, ok := meta.Annotations[keys[i]]
if values[i] == "" && !ok {
// don't set "" if annotation is not already there
continue
}
if err := rn.PipeE(yaml.SetAnnotation(keys[i], values[i])); err != nil {
func formatInternalAnnotations(rn *yaml.RNode, useInternal, useLegacy bool) error {
if !useInternal {
if err := rn.PipeE(yaml.ClearAnnotation(kioutil.IdAnnotation)); err != nil {
return err
}
if err := rn.PipeE(yaml.ClearAnnotation(kioutil.PathAnnotation)); err != nil {
return err
}
if err := rn.PipeE(yaml.ClearAnnotation(kioutil.IndexAnnotation)); err != nil {
return err
}
}
if !useLegacy {
if err := rn.PipeE(yaml.ClearAnnotation(kioutil.LegacyIdAnnotation)); err != nil {
return err
}
if err := rn.PipeE(yaml.ClearAnnotation(kioutil.LegacyPathAnnotation)); err != nil {
return err
}
if err := rn.PipeE(yaml.ClearAnnotation(kioutil.LegacyIndexAnnotation)); err != nil {
return err
}
}
return nil
}

View File

@@ -217,25 +217,9 @@ func TestLegacyAnnotationReconciliation(t *testing.T) {
}
return nodes, nil
}
changeLegacyId := func(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
changeBothPathAnnosMatch := func(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
for _, rn := range nodes {
if err := rn.PipeE(yaml.SetAnnotation(kioutil.LegacyIdAnnotation, "new")); err != nil {
return nil, err
}
}
return nodes, nil
}
changeInternalId := func(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
for _, rn := range nodes {
if err := rn.PipeE(yaml.SetAnnotation(kioutil.IdAnnotation, "new")); err != nil {
return nil, err
}
}
return nodes, nil
}
changeBothPathAnnos := func(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
for _, rn := range nodes {
if err := rn.PipeE(yaml.SetAnnotation(kioutil.LegacyPathAnnotation, "legacy")); err != nil {
if err := rn.PipeE(yaml.SetAnnotation(kioutil.LegacyPathAnnotation, "new")); err != nil {
return nil, err
}
if err := rn.PipeE(yaml.SetAnnotation(kioutil.PathAnnotation, "new")); err != nil {
@@ -244,6 +228,17 @@ func TestLegacyAnnotationReconciliation(t *testing.T) {
}
return nodes, nil
}
changeBothPathAnnosMismatch := func(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
for _, rn := range nodes {
if err := rn.PipeE(yaml.SetAnnotation(kioutil.LegacyPathAnnotation, "foo")); err != nil {
return nil, err
}
if err := rn.PipeE(yaml.SetAnnotation(kioutil.PathAnnotation, "bar")); err != nil {
return nil, err
}
}
return nodes, nil
}
noops := []Filter{
FilterFunc(noopFilter1),
@@ -251,9 +246,8 @@ func TestLegacyAnnotationReconciliation(t *testing.T) {
}
internal := []Filter{FilterFunc(changeInternalAnnos)}
legacy := []Filter{FilterFunc(changeLegacyAnnos)}
legacyId := []Filter{FilterFunc(changeLegacyId)}
internalId := []Filter{FilterFunc(changeInternalId)}
changeBoth := []Filter{FilterFunc(changeBothPathAnnos), FilterFunc(noopFilter1)}
changeBothMatch := []Filter{FilterFunc(changeBothPathAnnosMatch), FilterFunc(noopFilter1)}
changeBothMismatch := []Filter{FilterFunc(changeBothPathAnnosMismatch), FilterFunc(noopFilter1)}
testCases := map[string]struct {
input string
@@ -292,8 +286,6 @@ metadata:
annotations:
config.kubernetes.io/path: 'configmap.yaml'
config.kubernetes.io/index: '0'
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
@@ -304,8 +296,6 @@ metadata:
annotations:
config.kubernetes.io/path: "configmap.yaml"
config.kubernetes.io/index: '1'
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
@@ -341,8 +331,6 @@ metadata:
annotations:
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/index: '0'
config.kubernetes.io/path: 'configmap.yaml'
config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
@@ -353,15 +341,12 @@ metadata:
annotations:
internal.config.kubernetes.io/path: "configmap.yaml"
internal.config.kubernetes.io/index: '1'
config.kubernetes.io/path: 'configmap.yaml'
config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
},
// the orchestrator should detect that the legacy annotations
// have been changed by the function, and should update the
// new internal annotations to reflect the same change
// have been changed by the function
"change only legacy annotations": {
input: `apiVersion: v1
kind: ConfigMap
@@ -386,6 +371,190 @@ data:
filters: legacy,
expected: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
config.kubernetes.io/path: 'new'
config.kubernetes.io/index: 'new'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
config.kubernetes.io/path: "new"
config.kubernetes.io/index: 'new'
data:
grpcPort: 8081
`,
},
// the orchestrator should detect that the new internal annotations
// have been changed by the function
"change only internal annotations": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
internal.config.kubernetes.io/path: "configmap.yaml"
internal.config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
filters: internal,
expected: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
internal.config.kubernetes.io/path: 'new'
internal.config.kubernetes.io/index: 'new'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
internal.config.kubernetes.io/path: "new"
internal.config.kubernetes.io/index: 'new'
data:
grpcPort: 8081
`,
},
// the orchestrator should detect that the legacy annotations
// have been changed by the function
"change only internal annotations while input is legacy annotations": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
config.kubernetes.io/path: 'configmap.yaml'
config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
config.kubernetes.io/path: "configmap.yaml"
config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
filters: internal,
expected: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
config.kubernetes.io/path: 'new'
config.kubernetes.io/index: 'new'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
config.kubernetes.io/path: "new"
config.kubernetes.io/index: 'new'
data:
grpcPort: 8081
`,
},
// the orchestrator should detect that the new internal annotations
// have been changed by the function
"change only legacy annotations while input is internal annotations": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
internal.config.kubernetes.io/path: "configmap.yaml"
internal.config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
filters: legacy,
expected: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
internal.config.kubernetes.io/path: 'new'
internal.config.kubernetes.io/index: 'new'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
internal.config.kubernetes.io/path: "new"
internal.config.kubernetes.io/index: 'new'
data:
grpcPort: 8081
`,
},
// the orchestrator should detect that the legacy annotations
// have been changed by the function
"change only legacy annotations while input has both": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
config.kubernetes.io/path: 'configmap.yaml'
config.kubernetes.io/index: '0'
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
config.kubernetes.io/path: "configmap.yaml"
config.kubernetes.io/index: '1'
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
filters: legacy,
expected: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
@@ -410,16 +579,17 @@ data:
`,
},
// the orchestrator should detect that the new internal annotations
// have been changed by the function, and should update the
// legacy annotations to reflect the same change
"change only internal annotations": {
// have been changed by the function
"change only internal annotations while input has both": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
config.kubernetes.io/path: 'configmap.yaml'
config.kubernetes.io/path: "configmap.yaml"
config.kubernetes.io/index: '0'
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
@@ -430,6 +600,8 @@ metadata:
annotations:
config.kubernetes.io/path: "configmap.yaml"
config.kubernetes.io/index: '1'
internal.config.kubernetes.io/path: "configmap.yaml"
internal.config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
@@ -439,7 +611,7 @@ kind: ConfigMap
metadata:
name: ports-from
annotations:
config.kubernetes.io/path: 'new'
config.kubernetes.io/path: "new"
config.kubernetes.io/index: 'new'
internal.config.kubernetes.io/path: 'new'
internal.config.kubernetes.io/index: 'new'
@@ -453,86 +625,75 @@ metadata:
annotations:
config.kubernetes.io/path: "new"
config.kubernetes.io/index: 'new'
internal.config.kubernetes.io/path: 'new'
internal.config.kubernetes.io/path: "new"
internal.config.kubernetes.io/index: 'new'
data:
grpcPort: 8081
`,
},
// the orchestrator should detect that the new internal id annotation
// has been changed, and copy it over to the legacy one, and also
// copy the path and index legacy annotations to the new internal
// ones
"change only internal id when original legacy set": {
// the orchestrator should detect that the new internal annotations
// have been changed by the function
"change both to matching value while input has both": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
config.kubernetes.io/path: 'configmap.yaml'
config.kubernetes.io/path: "configmap.yaml"
config.kubernetes.io/index: '0'
config.k8s.io/id: '1'
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
config.kubernetes.io/path: "configmap.yaml"
config.kubernetes.io/index: '1'
internal.config.kubernetes.io/path: "configmap.yaml"
internal.config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
filters: internalId,
filters: changeBothMatch,
expected: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
config.kubernetes.io/path: 'configmap.yaml'
config.kubernetes.io/path: "new"
config.kubernetes.io/index: '0'
config.k8s.io/id: 'new'
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/path: 'new'
internal.config.kubernetes.io/index: '0'
internal.config.kubernetes.io/id: 'new'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
config.kubernetes.io/path: "new"
config.kubernetes.io/index: '1'
internal.config.kubernetes.io/path: "new"
internal.config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
},
// the orchestrator should detect that the legacy id annotation
// has been changed, and copy it over to the internal one, and also
// copy the path and index internal annotations to the legacy
// ones
"change only legacy id when internal legacy set": {
// the orchestrator should detect that the new internal annotations
// have been changed by the function
"change both to matching value while input is legacy": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/index: '0'
internal.config.kubernetes.io/id: '1'
data:
grpcPort: 8080
`,
filters: legacyId,
expected: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/index: '0'
internal.config.kubernetes.io/id: 'new'
config.kubernetes.io/path: 'configmap.yaml'
config.kubernetes.io/path: "configmap.yaml"
config.kubernetes.io/index: '0'
config.k8s.io/id: 'new'
data:
grpcPort: 8080
`,
},
// the function changes both the legacy and new path annotation,
// so we should get an error
"change both": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
config.kubernetes.io/path: 'configmap.yaml'
internal.kubernetes.io/path: 'configmap.yaml'
data:
grpcPort: 8080
---
@@ -546,7 +707,155 @@ metadata:
data:
grpcPort: 8081
`,
filters: changeBoth,
filters: changeBothMatch,
expected: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
config.kubernetes.io/path: "new"
config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
config.kubernetes.io/path: "new"
config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
},
// the orchestrator should detect that the new internal annotations
// have been changed by the function
"change both to matching value while input is internal": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
internal.config.kubernetes.io/path: 'configmap.yaml'
internal.config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
internal.config.kubernetes.io/path: "configmap.yaml"
internal.config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
filters: changeBothMatch,
expected: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
internal.config.kubernetes.io/path: 'new'
internal.config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
internal.config.kubernetes.io/path: "new"
internal.config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
},
// the function changes both the legacy and new path annotation, and they mismatch,
// so we should get an error
"change both but mismatch while input is legacy": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
config.kubernetes.io/path: 'configmap.yaml'
config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
config.kubernetes.io/path: "configmap.yaml"
config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
filters: changeBothMismatch,
expectedErr: "resource input to function has mismatched legacy and internal path annotations",
},
// the function changes both the legacy and new path annotation, and they mismatch,
// so we should get an error
"change both but mismatch while input is internal": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
internal.config.kubernetes.io/path: "configmap.yaml"
internal.config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
internal.config.kubernetes.io/path: "configmap.yaml"
internal.config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
filters: changeBothMismatch,
expectedErr: "resource input to function has mismatched legacy and internal path annotations",
},
// the function changes both the legacy and new path annotation, and they mismatch,
// so we should get an error
"change both but mismatch while input has both": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: ports-from
annotations:
config.kubernetes.io/path: 'configmap.yaml'
config.kubernetes.io/index: '0'
config.k8s.io/id: '1'
internal.config.kubernetes.io/path: "configmap.yaml"
internal.config.kubernetes.io/index: '0'
data:
grpcPort: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ports-to
annotations:
config.kubernetes.io/path: "configmap.yaml"
config.kubernetes.io/index: '1'
config.k8s.io/id: '2'
internal.config.kubernetes.io/path: "configmap.yaml"
internal.config.kubernetes.io/index: '1'
data:
grpcPort: 8081
`,
filters: changeBothMismatch,
expectedErr: "resource input to function has mismatched legacy and internal path annotations",
},
}

View File

@@ -41,21 +41,37 @@ const (
// Deprecated: use IdAnnotation instead.
LegacyIdAnnotation = "config.k8s.io/id"
// InternalAnnotationsMigrationResourceIDAnnotation is used to uniquely identify
// resources during round trip to and from a function execution. We will use it
// to track the internal annotations and reconcile them if needed.
InternalAnnotationsMigrationResourceIDAnnotation = internalPrefix + "annotations-migration-resource-id"
)
func GetFileAnnotations(rn *yaml.RNode) (string, string, error) {
if err := CopyLegacyAnnotations(rn); err != nil {
return "", "", err
rm, _ := rn.GetMeta()
annotations := rm.Annotations
path, found := annotations[PathAnnotation]
if !found {
path = annotations[LegacyPathAnnotation]
}
meta, err := rn.GetMeta()
if err != nil {
return "", "", err
index, found := annotations[IndexAnnotation]
if !found {
index = annotations[LegacyIndexAnnotation]
}
path := meta.Annotations[PathAnnotation]
index := meta.Annotations[IndexAnnotation]
return path, index, nil
}
func GetIdAnnotation(rn *yaml.RNode) string {
rm, _ := rn.GetMeta()
annotations := rm.Annotations
id, found := annotations[IdAnnotation]
if !found {
id = annotations[LegacyIdAnnotation]
}
return id
}
func CopyLegacyAnnotations(rn *yaml.RNode) error {
meta, err := rn.GetMeta()
if err != nil {
@@ -377,13 +393,16 @@ func ConfirmInternalAnnotationUnchanged(r1 *yaml.RNode, r2 *yaml.RNode, exclusio
return nil
}
// GetInternalAnnotations returns a map of all the annotations of the provided RNode that begin
// with the prefix `internal.config.kubernetes.io`
// GetInternalAnnotations returns a map of all the annotations of the provided
// RNode that satisfies one of the following: 1) begin with the prefix
// `internal.config.kubernetes.io` 2) is one of `config.kubernetes.io/path`,
// `config.kubernetes.io/index` and `config.k8s.io/id`.
func GetInternalAnnotations(rn *yaml.RNode) map[string]string {
annotations := rn.GetAnnotations()
meta, _ := rn.GetMeta()
annotations := meta.Annotations
result := make(map[string]string)
for k, v := range annotations {
if strings.HasPrefix(k, internalPrefix) {
if strings.HasPrefix(k, internalPrefix) || k == LegacyPathAnnotation || k == LegacyIndexAnnotation || k == LegacyIdAnnotation {
result[k] = v
}
}

View File

@@ -49,7 +49,7 @@ func ParseGroupVersion(apiVersion string) (group, version string) {
// GvkFromString makes a Gvk from the output of Gvk.String().
func GvkFromString(s string) Gvk {
values := strings.Split(s, fieldSep)
if len(values) != 3 {
if len(values) < 3 {
// ...then the string didn't come from Gvk.String().
return Gvk{
Group: noGroup,
@@ -57,27 +57,27 @@ func GvkFromString(s string) Gvk {
Kind: noKind,
}
}
g := values[0]
if g == noGroup {
g = ""
k := values[0]
if k == noKind {
k = ""
}
v := values[1]
if v == noVersion {
v = ""
}
k := values[2]
if k == noKind {
k = ""
g := strings.Join(values[2:], fieldSep)
if g == noGroup {
g = ""
}
return NewGvk(g, v, k)
}
// Values that are brief but meaningful in logs.
const (
noGroup = "~G"
noVersion = "~V"
noKind = "~K"
fieldSep = "_"
noGroup = "[noGrp]"
noVersion = "[noVer]"
noKind = "[noKind]"
fieldSep = "."
)
// String returns a string representation of the GVK.
@@ -94,7 +94,7 @@ func (x Gvk) String() string {
if k == "" {
k = noKind
}
return strings.Join([]string{g, v, k}, fieldSep)
return strings.Join([]string{k, v, g}, fieldSep)
}
// ApiVersion returns the combination of Group and Version
@@ -109,7 +109,8 @@ func (x Gvk) ApiVersion() string {
}
// StringWoEmptyField returns a string representation of the GVK. Non-exist
// fields will be omitted.
// fields will be omitted. This is called when generating a filename for the
// resource.
func (x Gvk) StringWoEmptyField() string {
var s []string
if x.Group != "" {
@@ -121,7 +122,7 @@ func (x Gvk) StringWoEmptyField() string {
if x.Kind != "" {
s = append(s, x.Kind)
}
return strings.Join(s, fieldSep)
return strings.Join(s, "_")
}
// Equals returns true if the Gvk's have equal fields.

View File

@@ -47,8 +47,8 @@ var lessThanTests = []struct {
Gvk{Group: "a", Version: "c", Kind: "ClusterRole"}},
{Gvk{Group: "a", Version: "c", Kind: "Namespace"},
Gvk{Group: "a", Version: "b", Kind: "ClusterRole"}},
{Gvk{Group: "a", Version: "d", Kind: "Namespace"},
Gvk{Group: "b", Version: "c", Kind: "Namespace"}},
{Gvk{Group: "b", Version: "c", Kind: "Namespace"},
Gvk{Group: "a", Version: "d", Kind: "Namespace"}},
{Gvk{Group: "a", Version: "b", Kind: orderFirst[len(orderFirst)-1]},
Gvk{Group: "a", Version: "b", Kind: orderLast[0]}},
{Gvk{Group: "a", Version: "b", Kind: orderFirst[len(orderFirst)-1]},
@@ -87,14 +87,16 @@ var stringTests = []struct {
s string
r string
}{
{Gvk{}, "~G_~V_~K", ""},
{Gvk{Kind: "k"}, "~G_~V_k", "k"},
{Gvk{Version: "v"}, "~G_v_~K", "v"},
{Gvk{Version: "v", Kind: "k"}, "~G_v_k", "v_k"},
{Gvk{Group: "g"}, "g_~V_~K", "g"},
{Gvk{Group: "g", Kind: "k"}, "g_~V_k", "g_k"},
{Gvk{Group: "g", Version: "v"}, "g_v_~K", "g_v"},
{Gvk{Group: "g", Version: "v", Kind: "k"}, "g_v_k", "g_v_k"},
{Gvk{}, "[noKind].[noVer].[noGrp]", ""},
{Gvk{Kind: "k"}, "k.[noVer].[noGrp]", "k"},
{Gvk{Version: "v"}, "[noKind].v.[noGrp]", "v"},
{Gvk{Version: "v", Kind: "k"}, "k.v.[noGrp]", "v_k"},
{Gvk{Group: "g"}, "[noKind].[noVer].g", "g"},
{Gvk{Group: "g", Kind: "k"}, "k.[noVer].g", "g_k"},
{Gvk{Group: "g", Version: "v"}, "[noKind].v.g", "g_v"},
{Gvk{Group: "g", Version: "v", Kind: "k"}, "k.v.g", "g_v_k"},
{Gvk{Group: "rbac.authorization.k8s.io", Version: "v1", Kind: "ClusterRole", isClusterScoped: true},
"ClusterRole.v1.rbac.authorization.k8s.io", "rbac.authorization.k8s.io_v1_ClusterRole"},
}
func TestString(t *testing.T) {

View File

@@ -37,9 +37,9 @@ func NewResIdKindOnly(k string, n string) ResId {
}
const (
noNamespace = "~X"
noName = "~N"
separator = "|"
noNamespace = "[noNs]"
noName = "[noName]"
separator = "/"
TotallyNotANamespace = "_non_namespaceable_"
DefaultNamespace = "default"
)
@@ -55,33 +55,31 @@ func (id ResId) String() string {
nm = noName
}
return strings.Join(
[]string{id.Gvk.String(), ns, nm}, separator)
[]string{id.Gvk.String(), strings.Join([]string{nm, ns}, fieldSep)}, separator)
}
func FromString(s string) ResId {
values := strings.Split(s, separator)
g := GvkFromString(values[0])
gvk := GvkFromString(values[0])
ns := values[1]
values = strings.Split(values[1], fieldSep)
last := len(values)-1
ns := values[last]
if ns == noNamespace {
ns = ""
}
nm := values[2]
nm := strings.Join(values[:last], fieldSep)
if nm == noName {
nm = ""
}
return ResId{
Gvk: g,
Gvk: gvk,
Namespace: ns,
Name: nm,
}
}
// GvknString of ResId based on GVK and name
func (id ResId) GvknString() string {
return id.Gvk.String() + separator + id.Name
}
// GvknEquals returns true if the other id matches
// Group/Version/Kind/name.
func (id ResId) GvknEquals(o ResId) bool {

View File

@@ -17,7 +17,7 @@ var resIdStringTests = []struct {
Gvk: Gvk{Group: "g", Version: "v", Kind: "k"},
Name: "nm",
},
"g_v_k|ns|nm",
"k.v.g/nm.ns",
},
{
ResId{
@@ -25,7 +25,7 @@ var resIdStringTests = []struct {
Gvk: Gvk{Version: "v", Kind: "k"},
Name: "nm",
},
"~G_v_k|ns|nm",
"k.v.[noGrp]/nm.ns",
},
{
ResId{
@@ -33,7 +33,7 @@ var resIdStringTests = []struct {
Gvk: Gvk{Kind: "k"},
Name: "nm",
},
"~G_~V_k|ns|nm",
"k.[noVer].[noGrp]/nm.ns",
},
{
ResId{
@@ -41,38 +41,26 @@ var resIdStringTests = []struct {
Gvk: Gvk{},
Name: "nm",
},
"~G_~V_~K|ns|nm",
"[noKind].[noVer].[noGrp]/nm.ns",
},
{
ResId{
Gvk: Gvk{},
Name: "nm",
},
"~G_~V_~K|~X|nm",
},
{
ResId{
Gvk: Gvk{},
Name: "nm",
},
"~G_~V_~K|~X|nm",
"[noKind].[noVer].[noGrp]/nm.[noNs]",
},
{
ResId{
Gvk: Gvk{},
},
"~G_~V_~K|~X|~N",
},
{
ResId{
Gvk: Gvk{},
},
"~G_~V_~K|~X|~N",
"[noKind].[noVer].[noGrp]/[noName].[noNs]",
},
{
ResId{},
"~G_~V_~K|~X|~N",
"[noKind].[noVer].[noGrp]/[noName].[noNs]",
},
}
func TestResIdString(t *testing.T) {
@@ -83,84 +71,7 @@ func TestResIdString(t *testing.T) {
}
}
var gvknStringTests = []struct {
x ResId
s string
}{
{
ResId{
Namespace: "ns",
Gvk: Gvk{Group: "g", Version: "v", Kind: "k"},
Name: "nm",
},
"g_v_k|nm",
},
{
ResId{
Namespace: "ns",
Gvk: Gvk{Version: "v", Kind: "k"},
Name: "nm",
},
"~G_v_k|nm",
},
{
ResId{
Namespace: "ns",
Gvk: Gvk{Kind: "k"},
Name: "nm",
},
"~G_~V_k|nm",
},
{
ResId{
Namespace: "ns",
Gvk: Gvk{},
Name: "nm",
},
"~G_~V_~K|nm",
},
{
ResId{
Gvk: Gvk{},
Name: "nm",
},
"~G_~V_~K|nm",
},
{
ResId{
Gvk: Gvk{},
Name: "nm",
},
"~G_~V_~K|nm",
},
{
ResId{
Gvk: Gvk{},
},
"~G_~V_~K|",
},
{
ResId{
Gvk: Gvk{},
},
"~G_~V_~K|",
},
{
ResId{},
"~G_~V_~K|",
},
}
func TestGvknString(t *testing.T) {
for _, hey := range gvknStringTests {
if hey.x.GvknString() != hey.s {
t.Fatalf("Actual: %s, Expected: '%s'", hey.x.GvknString(), hey.s)
}
}
}
func TestResIdEquals(t *testing.T) {
var GvknEqualsTest = []struct {
id1 ResId
id2 ResId
@@ -358,6 +269,24 @@ var ids = []ResId{
{
Gvk: Gvk{},
},
{
Gvk: Gvk{
Group: "rbac.authorization.k8s.io",
Version: "v1",
Kind: "ClusterRole",
isClusterScoped: true,
},
Name: "nm",
},
{
Gvk: Gvk{
Group: "rbac.authorization.k8s.io",
Version: "v1",
Kind: "ClusterRole",
isClusterScoped: true,
},
Name: "my.name",
},
}
func TestResIdIsSelected(t *testing.T) {

View File

@@ -1063,7 +1063,6 @@ func checkKey(key string, elems []*Node) bool {
return count == len(elems)
}
// Deprecated: use pipes instead.
// GetSlice returns the contents of the slice field at the given path.
func (rn *RNode) GetSlice(path string) ([]interface{}, error) {
value, err := rn.GetFieldValue(path)
@@ -1076,7 +1075,6 @@ func (rn *RNode) GetSlice(path string) ([]interface{}, error) {
return nil, fmt.Errorf("node %s is not a slice", path)
}
// Deprecated: use pipes instead.
// GetString returns the contents of the string field at the given path.
func (rn *RNode) GetString(path string) (string, error) {
value, err := rn.GetFieldValue(path)
@@ -1089,7 +1087,6 @@ func (rn *RNode) GetString(path string) (string, error) {
return "", fmt.Errorf("node %s is not a string: %v", path, value)
}
// Deprecated: use slash paths instead.
// GetFieldValue finds period delimited fields.
// TODO: When doing kustomize var replacement, which is likely a
// a primary use of this function and the reason it returns interface{}

View File

@@ -1,4 +1,4 @@
// Copyright 2019 The Kubernetes Authors.
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
//go:generate pluginator
@@ -7,26 +7,24 @@ package main
import (
"errors"
"sigs.k8s.io/kustomize/api/filters/prefixsuffix"
"sigs.k8s.io/kustomize/api/filters/prefix"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/yaml"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
// Add the given prefix and suffix to the field.
// Add the given prefix to the field
type plugin struct {
Prefix string `json:"prefix,omitempty" yaml:"prefix,omitempty"`
Suffix string `json:"suffix,omitempty" yaml:"suffix,omitempty"`
FieldSpecs types.FsSlice `json:"fieldSpecs,omitempty" yaml:"fieldSpecs,omitempty"`
}
//noinspection GoUnusedGlobalVariable
var KustomizePlugin plugin
// A Gvk skip list for prefix/suffix modification.
// hard coded for now - eventually should be part of config.
var prefixSuffixFieldSpecsToSkip = types.FsSlice{
// TODO: Make this gvk skip list part of the config.
var prefixFieldSpecsToSkip = types.FsSlice{
{Gvk: resid.Gvk{Kind: "CustomResourceDefinition"}},
{Gvk: resid.Gvk{Group: "apiregistration.k8s.io", Kind: "APIService"}},
{Gvk: resid.Gvk{Kind: "Namespace"}},
@@ -35,7 +33,6 @@ var prefixSuffixFieldSpecsToSkip = types.FsSlice{
func (p *plugin) Config(
_ *resmap.PluginHelpers, c []byte) (err error) {
p.Prefix = ""
p.Suffix = ""
p.FieldSpecs = nil
err = yaml.Unmarshal(c, p)
if err != nil {
@@ -48,9 +45,9 @@ func (p *plugin) Config(
}
func (p *plugin) Transform(m resmap.ResMap) error {
// Even if both the Prefix and Suffix are empty we want
// to proceed with the transformation. This allows to add contextual
// information to the resources (AddNamePrefix and AddNameSuffix).
// Even if the Prefix is empty we want to proceed with the
// transformation. This allows to add contextual information
// to the resources (AddNamePrefix).
for _, r := range m.Resources() {
// TODO: move this test into the filter (i.e. make a better filter)
if p.shouldSkip(r.OrgId()) {
@@ -65,21 +62,21 @@ func (p *plugin) Transform(m resmap.ResMap) error {
continue
}
// TODO: move this test into the filter.
if smellsLikeANameChange(&fs) {
if fs.Path == "metadata/name" {
// "metadata/name" is the only field.
// this will add a prefix and a suffix
// to the resource even if those are
// empty
// this will add a prefix to the resource
// even if it is empty
r.AddNamePrefix(p.Prefix)
r.AddNameSuffix(p.Suffix)
if p.Prefix != "" || p.Suffix != "" {
if p.Prefix != "" {
// TODO: There are multiple transformers that can change a resource's name, and each makes a call to
// StorePreviousID(). We should make it so that we only call StorePreviousID once per kustomization layer
// to avoid storing intermediate names between transformations, to prevent intermediate name conflicts.
r.StorePreviousId()
}
}
if err := r.ApplyFilter(prefixsuffix.Filter{
if err := r.ApplyFilter(prefix.Filter{
Prefix: p.Prefix,
Suffix: p.Suffix,
FieldSpec: fs,
}); err != nil {
return err
@@ -89,12 +86,8 @@ func (p *plugin) Transform(m resmap.ResMap) error {
return nil
}
func smellsLikeANameChange(fs *types.FieldSpec) bool {
return fs.Path == "metadata/name"
}
func (p *plugin) shouldSkip(id resid.ResId) bool {
for _, path := range prefixSuffixFieldSpecsToSkip {
for _, path := range prefixFieldSpecsToSkip {
if id.IsSelected(&path.Gvk) {
return true
}

View File

@@ -1,4 +1,4 @@
// Copyright 2019 The Kubernetes Authors.
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package main_test
@@ -9,18 +9,17 @@ import (
kusttest_test "sigs.k8s.io/kustomize/api/testutils/kusttest"
)
func TestPrefixSuffixTransformer(t *testing.T) {
func TestPrefixTransformer(t *testing.T) {
th := kusttest_test.MakeEnhancedHarness(t).
PrepBuiltin("PrefixSuffixTransformer")
PrepBuiltin("PrefixTransformer")
defer th.Reset()
rm := th.LoadAndRunTransformer(`
apiVersion: builtin
kind: PrefixSuffixTransformer
kind: PrefixTransformer
metadata:
name: notImportantHere
prefix: baked-
suffix: -pie
fieldSpecs:
- path: metadata/name
`, `
@@ -67,8 +66,7 @@ metadata:
internal.config.kubernetes.io/previousKinds: Service
internal.config.kubernetes.io/previousNames: apple
internal.config.kubernetes.io/previousNamespaces: default
internal.config.kubernetes.io/suffixes: -pie
name: baked-apple-pie
name: baked-apple
spec:
ports:
- port: 7002
@@ -91,13 +89,12 @@ metadata:
internal.config.kubernetes.io/previousKinds: ConfigMap
internal.config.kubernetes.io/previousNames: cm
internal.config.kubernetes.io/previousNamespaces: default
internal.config.kubernetes.io/suffixes: -pie
name: baked-cm-pie
name: baked-cm
`)
rm = th.LoadAndRunTransformer(`
apiVersion: builtin
kind: PrefixSuffixTransformer
kind: PrefixTransformer
metadata:
name: notImportantHere
prefix: test-

Some files were not shown because too many files have changed in this diff Show More