Compare commits

...

129 Commits

Author SHA1 Message Date
Jeff Regan
a0c7997b66 Merge pull request #3963 from natasha41575/KustomizeEditFix
small `api` changes for kustomize fix
2021-06-08 13:52:06 -07:00
Natasha Sarkar
7458a53a73 copy method for selector 2021-06-08 11:18:38 -07:00
Natasha Sarkar
cf6e6ca4db omitempty for replacement type 2021-06-08 11:18:27 -07:00
Natasha Sarkar
e847ec7474 ReadDir method for filesys 2021-06-08 11:18:11 -07:00
Jeff Regan
440026b9b3 Merge pull request #3965 from mengqiy/listcomments
handle comments in list correctly
2021-06-07 17:32:53 -07:00
Kubernetes Prow Robot
64331ad845 Merge pull request #3964 from ash2k/ash2k/handle-errors2
Handle errors
2021-06-07 16:26:03 -07:00
Mengqi Yu
294070b3ab address comments 2021-06-07 15:06:57 -07:00
Mengqi Yu
cabbea0d97 handle comments in list correctly 2021-06-07 14:57:49 -07:00
Jeff Regan
732a8522df Merge pull request #3968 from monopole/fixFailingHelmTest
Don't run helm inflator test against released version.
2021-06-07 13:19:02 -07:00
monopole
8f82c4c748 Don't run helm inflator test against released version. 2021-06-07 12:26:12 -07:00
Jeff Regan
d0bc25f339 Update byteio_writer.go 2021-06-07 12:25:04 -07:00
Jeff Regan
ed3200e4f5 Remove bad TODO 2021-06-04 17:56:42 -07:00
Mikhail Mazurskiy
a3ed120efb Handle errors 2021-06-05 09:43:13 +10:00
Jeff Regan
f1b191c02f Update README.md 2021-06-04 16:22:59 -07:00
Jeff Regan
1493b24b46 Update README.md 2021-06-04 16:21:25 -07:00
Kubernetes Prow Robot
5993eae1aa Merge pull request #3934 from yhrn/fix-json-sink
Sink: Force JSON encoding for .json files
2021-06-04 11:38:38 -07:00
Mattias Öhrn
3e506eae02 PR feedback - more tests and some cleanup 2021-06-04 11:06:22 +02:00
Kubernetes Prow Robot
0305860078 Merge pull request #3962 from natasha41575/resourceRef
remove metadata field from resourceRef schema
2021-06-03 15:57:02 -07:00
Natasha Sarkar
0205090e0d remove metadata field from resourceRef schema 2021-06-03 12:46:51 -07:00
Jeff Regan
da1bd901b4 Merge pull request #3939 from KnVerey/framework_test_helpers
[kyaml] Improvements to frameworktestutil
2021-06-03 11:14:54 -07:00
Jeff Regan
636b9c7aeb Merge pull request #3931 from natasha41575/ReplacementOverwritesSource
Fix issues with replacements
2021-06-03 10:57:01 -07:00
Mattias Öhrn
942f112ef5 Fix kyaml tests 2021-06-03 08:31:03 +02:00
Mattias Öhrn
03bbb076bf plugin/builtin/iampolicygenerator/go.sum tidy 2021-06-03 08:30:38 +02:00
Mattias Öhrn
e468d6b4d2 Fixing cmd/config/internal/commands/e2e/e2e_test.go 2021-06-03 08:29:53 +02:00
Kubernetes Prow Robot
57206a628d Merge pull request #3955 from mengqiy/betterresults
Check for empty GKNN when formatting results
2021-06-02 18:05:38 -07:00
Mengqi Yu
f061bb887b Check for empty GKNN when formatting results 2021-06-02 16:32:18 -07:00
Kubernetes Prow Robot
75fd9a43a3 Merge pull request #3925 from frankfarzan/krm-functions-spec
Fully specify KRM Functions Spec and graduate it to v1.
2021-06-02 12:10:42 -07:00
Frank Farzan
58165dfc89 comments 2021-06-02 10:53:49 -07:00
Frank Farzan
0e8257c387 comments 2021-06-02 10:50:39 -07:00
Kubernetes Prow Robot
62e78f8349 Merge pull request #3940 from Shell32-Natsu/resource-factory
add an option to include local configs
2021-06-01 17:20:12 -07:00
Natasha Sarkar
84724a3ebf smarter path splitter for replacements 2021-06-01 17:15:54 -07:00
Donny Xia
23544e0431 code review 2021-06-01 16:25:41 -07:00
Donny Xia
b1fda3d62e add an option to include local configs 2021-06-01 13:38:36 -07:00
Natasha Sarkar
b8ae69b748 copy target rnode in replacements 2021-06-01 13:24:51 -07:00
Natasha Sarkar
4014440d06 test to demonstrate '.' in list element path issue 2021-06-01 13:24:51 -07:00
Natasha Sarkar
74b0b3adc6 test to demonstrate multiple fieldpaths issue in replacements 2021-06-01 13:24:51 -07:00
Katrina Verey
382f09a126 Make frameworktestutil assertions configurable
Also refactor to share common code between results checkers
2021-06-01 12:20:23 -07:00
Katrina Verey
f9afdc5c95 Improve frameworktestutil usability with complex error messages 2021-06-01 12:20:17 -07:00
Mattias Öhrn
5e4fb4796e Only encoding non-wrapped single .json items as JSON 2021-06-01 20:44:21 +02:00
Katrina Verey
76f8988865 Mark updated tests as skipped instead of passed 2021-05-31 10:56:32 -07:00
Mattias Öhrn
fa3e829eb6 Sink: Force JSON encoding for .json files 2021-05-31 08:00:54 +02:00
Jeff Regan
d9435bd1b1 Merge pull request #3898 from dmizelle/add-helm-include-crds
Add includeCRDs Field to HelmChart
2021-05-27 16:04:07 -07:00
Kubernetes Prow Robot
af96bb4bda Merge pull request #3914 from natasha41575/GkeServiceAccountGenerator
Gke service account generator
2021-05-27 15:36:23 -07:00
Kubernetes Prow Robot
8607e0adec Merge pull request #3930 from monopole/demo3929
Demonstrate issue 3929
2021-05-26 17:04:06 -07:00
Natasha Sarkar
5a2a7709a4 add IAMPolicyGenerator 2021-05-26 16:54:38 -07:00
monopole
437e8f90f6 Demonstrate issue 3929 2021-05-26 16:29:55 -07:00
Jeff Regan
06ac670951 Merge pull request #3911 from ash2k/ash2k/handle-errors
Handle errors
2021-05-25 21:05:26 -07:00
Frank Farzan
3ee1579688 Fully specify KRM Functions Spec and graduate it to v1.
- Define the OpenAPI schema for ResourceList
- Graduate `ResourceList` to apiVersion `v1`. Many functions have been implemented
  and we are effectively treating the spec as backwards compatible.
- Add `results` field to the spec. This has been implemented in the
  orchestrator and couple of functions libraries for a while, but hasn't been
  yet standardized leading to minor discrepencies. Concurrent to this PR,
  we need to udpate the fn framework (kyaml) to be compatible with
  the standard and update the orchestrator to be backwards compatible.
- Include per-resource annotations (previously a seperate doc)
- Remove support for non-ResourceList wire formats. In practice, this is
  not widely used and complicates the standard. The orchestrators can
  easily covert a List to a ResourceList.
2021-05-25 16:09:32 -07:00
Jeff Regan
5954314b98 Merge pull request #3917 from zhouhao3/clean-tempfile
Cleanup tempfiles
2021-05-24 20:01:33 -07:00
Jeff Regan
c0324456a7 Merge pull request #3921 from monopole/deleteScriptsDir
Consolidate scripts into k8s-traditional hack dir.
2021-05-24 17:25:01 -07:00
monopole
172adc404f Consolidate scripts into k8s-traditional hack dir. 2021-05-24 17:09:11 -07:00
Jeff Regan
501748192b Merge pull request #3920 from monopole/unpinEverything
Back to development mode; unpin the modules
2021-05-24 09:42:57 -07:00
monopole
f6e6ac0320 Back to development mode; unpin the modules 2021-05-24 09:30:04 -07:00
Zhou Hao
a10ce1d787 cleanup tempfiles for runfn_test
Signed-off-by: Zhou Hao <zhouhao@fujitsu.com>
2021-05-24 17:01:44 +08:00
Zhou Hao
839cc2467c cleanup tempfiles for fmtr_test
Signed-off-by: Zhou Hao <zhouhao@fujitsu.com>
2021-05-24 15:52:28 +08:00
Mikhail Mazurskiy
dbc11ed29f Handle errors 2021-05-21 16:53:14 +10:00
Jeff Regan
0f614e92f7 Merge pull request #3909 from monopole/pinToApi
Pin to api v0.8.10
2021-05-20 13:44:39 -07:00
monopole
afaf7c62bc Pin to api v0.8.10 2021-05-20 13:34:27 -07:00
Jeff Regan
78d22069d7 Merge pull request #3910 from monopole/noMorePluginPin
No more plugin pinning.
2021-05-20 13:32:25 -07:00
monopole
22720a8b7a No more plugin pinning. 2021-05-20 13:31:50 -07:00
Jeff Regan
38c66d213a Update README.md 2021-05-20 13:14:28 -07:00
Jeff Regan
871de80544 Merge pull request #3908 from monopole/pinToCmdConfig
Pin to cmd/config v0.9.12
2021-05-20 13:05:20 -07:00
monopole
c24daec480 Pin to cmd/config v0.9.12 2021-05-20 12:54:56 -07:00
Jeff Regan
0849d12572 Update README.md 2021-05-20 12:54:06 -07:00
Jeff Regan
23bd8ff749 Merge pull request #3907 from monopole/fixFormatting
Fix go.mod in plugins.
2021-05-20 12:53:00 -07:00
monopole
b5759305af Fix go.mod in plugins. 2021-05-20 12:52:34 -07:00
Jeff Regan
da518668b5 Merge pull request #3906 from monopole/pinToKyaml
Pin to kyaml v0.10.20
2021-05-20 12:46:05 -07:00
monopole
51605beb3b Pin to kyaml v0.10.20 2021-05-20 12:35:01 -07:00
Jeff Regan
2646861a4c Update README.md 2021-05-20 12:29:15 -07:00
Jeff Regan
f8c910bd3b Update go.mod 2021-05-20 12:27:03 -07:00
Jeff Regan
cb82dced8a Merge pull request #3905 from monopole/fixMakefile
Fix makefile install-tools for Go 1.16
2021-05-20 11:12:12 -07:00
monopole
f618f9ce96 Fix makefile install-tools for Go 1.16 2021-05-20 10:54:13 -07:00
Jeff Regan
020dc9c216 Merge pull request #3903 from monopole/fixRepoSpecTest
Fix repospec test
2021-05-20 09:54:12 -07:00
monopole
a6b9445702 Fix repospec test 2021-05-20 09:52:59 -07:00
Jeff Regan
36408a120c Update HelmChartInflationGenerator.go 2021-05-19 17:34:44 -07:00
Jeff Regan
c1bca9cd62 Merge pull request #3900 from joshdk/master
URL based configuration for git exec timeouts and git submodule cloning
2021-05-19 16:10:35 -07:00
Jeff Regan
701973b73e Merge pull request #3899 from KnVerey/embedfs
[kyaml fn framework] Swap pkger for embed.FS compatibility
2021-05-19 09:29:23 -07:00
Josh Komoroske
24a64bdee3 URL based configuration for git exec timeouts and git submodule cloning
Adds a number of user-accessable options for configuring internal git resource
cloning behavior.
- Git commands are executed with a configurable timeout by including a parameter
  like "?timeout=2m30s" in the resource URL. This can improve cloning a large
  repository, or over a slow network.
- Git submodule cloning can be disabled by including a parameter like
  "?submodules=false" in the resource URL.
- Switch the overall query parsing to use url.Parse() and be more extensible.
2021-05-19 00:07:02 -07:00
Katrina Verey
3f3d3b17a4 Replace pkger with embed.FS compatibility 2021-05-18 12:07:42 -07:00
Katrina Verey
53c87a32e9 Reset openapi in Filter and show use of pkger with filepath 2021-05-18 11:20:42 -07:00
Katrina Verey
0fdf0f825f TemplateProcessor can add custom resource schemas to openapi 2021-05-18 11:20:42 -07:00
Devon Mizelle
73da51d0ac Apply suggestions from code review
Co-authored-by: Steven E. Harris <seh@panix.com>
2021-05-17 12:18:33 -04:00
Devon Mizelle
df10d5a17d Add includeCRDs Field to HelmChart
This commit adds functionality for a user to specify that `helm` should
include CRDs when inflating a Helm Chart.

As of Helm v3, the `install-crd` hook is no more, with
`CustomResourceDefinitions` existing in the root of the chart, under the
`crds` directory.

When calling `helm template`, `helm` does not output these CRDs unless
the user passes the `--include-crds` flag to the command.

With this commit, users can set `includeCRDs: true` as part of their helm
chart definition in `kustomize.yaml` to have these included as part of
the output.

Signed-off-by: Devon Mizelle <devon.mizelle@onepeloton.com>
2021-05-14 16:46:30 -04:00
Jeff Regan
9557888b32 Merge pull request #3889 from sylr/fix-tests-macos
Fix api tests on MacOS
2021-05-13 15:22:18 -07:00
Jeff Regan
3e14a31312 Merge pull request #3893 from natasha41575/fixTests
use assert statements for kustomize edit tests
2021-05-12 16:33:27 -07:00
Natasha Sarkar
dca13a4770 use assert statements for kustomize edit tests 2021-05-12 16:20:12 -07:00
Kubernetes Prow Robot
017a094438 Merge pull request #3888 from monopole/popRNodeUp
Remove delegation to RNode in Resource
2021-05-11 13:35:14 -07:00
Kubernetes Prow Robot
b8e7cf04b6 Merge pull request #3832 from cehoffman/helm-chart-namespace
Add ability to set target namespace for helm chart template
2021-05-11 13:09:13 -07:00
Sylvain Rabot
a8dacdaffc Fix api tests on MacOS
On MacOS /var is a symlink for /private/var and we can end up having the
loader having a /private/var path while the TMPDIR has a /var path which
triggers a panic.

Signed-off-by: Sylvain Rabot <sylvain@abstraction.fr>
2021-05-11 20:02:56 +02:00
monopole
5c4e363f11 Remove delegation to RNode in Resource. 2021-05-11 10:49:17 -07:00
Kubernetes Prow Robot
1e3ce57077 Merge pull request #3886 from monopole/applyFilter
Introduce resmap.ApplyFilter.
2021-05-11 08:31:36 -07:00
monopole
01ddeb476d Introduce resmap.ApplyFilter. 2021-05-10 20:47:57 -07:00
Jeff Regan
714af0cd66 Merge pull request #3884 from monopole/nitterFix
Simplify kind and name change code.
2021-05-09 16:21:52 -07:00
monopole
82abd7e9ea Simplify kind and name change code. 2021-05-09 16:20:39 -07:00
Jeff Regan
823ff2d048 Update go.mod 2021-05-08 08:02:28 -07:00
Kubernetes Prow Robot
fcfdf6be51 Merge pull request #3880 from Shell32-Natsu/func-filter-result
Make result in function filter public
2021-05-06 15:43:02 -07:00
Donny Xia
627118c438 skip tests instead of comment out 2021-05-06 12:33:17 -07:00
Donny Xia
5bb7364967 Fix linter error 2021-05-06 09:54:35 -07:00
Donny Xia
a46926c1eb Disable tests that use non-exist image 2021-05-06 09:54:26 -07:00
Donny Xia
4f760a0850 make result public in function filter 2021-05-06 09:45:11 -07:00
Jeff Regan
e905411207 Update go.mod 2021-05-05 13:02:57 -07:00
Jeff Regan
1eb77a6cab Update README.md 2021-05-05 08:27:58 -07:00
Jeff Regan
8b1704bcf3 Merge pull request #3874 from monopole/depprobcheck
Add a dependency debugging directory.
2021-05-04 22:21:33 -07:00
monopole
a0edb2d966 Add a dependency debugging directory. 2021-05-04 22:11:52 -07:00
Kubernetes Prow Robot
02dff45d7d Merge pull request #3870 from Shell32-Natsu/fix-image-helm
add helm fields to ordered fields
2021-05-04 06:25:13 -07:00
Donny Xia
3cf18adae9 fix test 2021-05-03 17:02:55 -07:00
Donny Xia
2bec25b46e add new fields to ordered fields 2021-05-03 16:57:27 -07:00
Kubernetes Prow Robot
8ee308d5d6 Merge pull request #3863 from monopole/simplifyGvk
Simplify gvk, speed up cluster-scoped checks.
2021-05-03 16:06:20 -07:00
Jeff Regan
3c3c97f9b5 Merge pull request #3854 from natasha41575/updateOpenApiFetch
update openapi fetch command
2021-05-03 12:20:26 -07:00
Kubernetes Prow Robot
c3e8f6008e Merge pull request #3838 from monopole/helmNamespace
A last-mile helm example, with namespace and prefix
2021-05-03 11:02:08 -07:00
monopole
660847225d Simplify gvk, speed up cluster-scoped checks. 2021-05-02 13:17:33 -07:00
Jeff Regan
48d16f877b Merge pull request #3864 from monopole/runMoreTests
Run e2e tests against 4.1.2
2021-05-02 13:17:04 -07:00
monopole
a4f4945455 Run e2e tests against 4.1.2 2021-05-02 13:01:24 -07:00
Jeff Regan
bd7229ea17 Merge pull request #3862 from monopole/updateGoSums
Update plugin go.mod, go.sum w/r to kyaml.
2021-05-02 10:38:22 -07:00
monopole
72441ce3ef Update plugin go.mod, go.sum w/r to kyaml. 2021-05-02 10:22:22 -07:00
Jeff Regan
fe5b7a3b41 Merge pull request #3861 from monopole/backToDevMode
Require kyaml v0.10.19, and unpin everything.
2021-04-30 23:01:18 -07:00
monopole
a4db686b6c Unpin everything. 2021-04-30 22:49:38 -07:00
Jeff Regan
10026758d3 Merge pull request #3860 from monopole/moveResidFromApiToKyamlModule
Move resid from api to kyaml module
2021-04-30 22:14:24 -07:00
monopole
c8dddac5b9 Move resid package from api to kyaml 2021-04-30 20:39:32 -07:00
monopole
5a8a4d47a5 More pinning. 2021-04-30 20:21:36 -07:00
monopole
6c35c06f3e Pin the plugins. 2021-04-30 20:11:14 -07:00
monopole
1235047742 Establish pin state. 2021-04-30 19:32:43 -07:00
Jeff Regan
6c041c3a79 Merge pull request #3859 from monopole/pinToCmdConfig
Pin to cmd/config v0.9.11
2021-04-30 18:57:12 -07:00
monopole
1e7260b69a Pin to cmd/config v0.9.11 2021-04-30 18:46:59 -07:00
Jeff Regan
1de5fe608f Merge pull request #3858 from monopole/gorepoModAdjustments
In module lists, handle allowed replacements.
2021-04-30 18:36:25 -07:00
monopole
6c9bf58e7f In module lists, handle allowed replacements. 2021-04-30 18:35:59 -07:00
Natasha Sarkar
45fc67062e update openapi fetch command 2021-04-30 16:00:19 -07:00
monopole
d03a5ab95f Add another last mile helm example. 2021-04-23 11:10:45 -07:00
Chris Hoffman
53f78260a9 Add ability to set target namespace for helm chart template 2021-04-22 18:31:26 -05:00
272 changed files with 6324 additions and 2889 deletions

View File

@@ -23,7 +23,7 @@ jobs:
uses: actions/checkout@v2
- name: Lint
run: ./scripts/kyaml-pre-commit.sh
run: ./hack/kyaml-pre-commit.sh
env:
KUSTOMIZE_DOCKER_E2E: false # don't need to do e2e tests for linting

View File

@@ -22,58 +22,48 @@ REPO_NAME := "kustomize"
endif
.PHONY: all
all: verify-kustomize
all: install-tools verify-kustomize
.PHONY: verify-kustomize
verify-kustomize: \
lint-kustomize \
test-unit-kustomize-all \
test-examples-kustomize-against-HEAD \
test-examples-kustomize-against-4.0
test-examples-kustomize-against-4.1
# The following target referenced by a file in
# https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes-sigs/kustomize
.PHONY: prow-presubmit-check
prow-presubmit-check: \
install-tools \
lint-kustomize \
test-multi-module \
test-unit-kustomize-all \
test-unit-cmd-all \
test-go-mod \
test-examples-kustomize-against-HEAD \
test-examples-kustomize-against-4.0
test-examples-kustomize-against-4.1
.PHONY: verify-kustomize-e2e
verify-kustomize-e2e: test-examples-e2e-kustomize
# Other builds in this repo might want a different linter version.
# Without one Makefile to rule them all, the different makes
# cannot assume that golanci-lint is at the version they want
# since everything uses the same implicit GOPATH.
# This installs in a temp dir to avoid overwriting someone else's
# linter, then installs in MYGOBIN with a new name.
# Version pinned by hack/go.mod
# cannot assume that golanci-lint is at the version they want.
# This installs what kustomize wants to use.
$(MYGOBIN)/golangci-lint-kustomize:
( \
set -e; \
cd hack; \
GO111MODULE=on go build -tags=tools -o $(MYGOBIN)/golangci-lint-kustomize github.com/golangci/golangci-lint/cmd/golangci-lint; \
)
rm -f $(CURDIR)/hack/golangci-lint
GOBIN=$(CURDIR)/hack go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.23.8
mv $(CURDIR)/hack/golangci-lint $(MYGOBIN)/golangci-lint-kustomize
# Install from version specified in api/go.mod.
$(MYGOBIN)/mdrip:
cd api; \
go install github.com/monopole/mdrip
go install github.com/monopole/mdrip@v1.0.2
# Install from version specified in api/go.mod.
$(MYGOBIN)/stringer:
cd api; \
go install golang.org/x/tools/cmd/stringer
go get golang.org/x/tools/cmd/stringer
# Install from version specified in api/go.mod.
$(MYGOBIN)/goimports:
cd api; \
go install golang.org/x/tools/cmd/goimports
go get golang.org/x/tools/cmd/goimports
# Build from local source.
$(MYGOBIN)/gorepomod:
@@ -141,6 +131,7 @@ pSrc=plugin/builtin
_builtinplugins = \
AnnotationsTransformer.go \
ConfigMapGenerator.go \
IAMPolicyGenerator.go \
HashTransformer.go \
ImageTagTransformer.go \
LabelTransformer.go \
@@ -168,6 +159,7 @@ builtinplugins = $(patsubst %,$(pGen)/%,$(_builtinplugins))
# that file, will be recreated.
$(pGen)/AnnotationsTransformer.go: $(pSrc)/annotationstransformer/AnnotationsTransformer.go
$(pGen)/ConfigMapGenerator.go: $(pSrc)/configmapgenerator/ConfigMapGenerator.go
$(pGen)/GkeSaGenerator.go: $(pSrc)/gkesagenerator/GkeSaGenerator.go
$(pGen)/HashTransformer.go: $(pSrc)/hashtransformer/HashTransformer.go
$(pGen)/ImageTagTransformer.go: $(pSrc)/imagetagtransformer/ImageTagTransformer.go
$(pGen)/LabelTransformer.go: $(pSrc)/labeltransformer/LabelTransformer.go
@@ -211,7 +203,7 @@ clean-kustomize-external-go-plugin:
### End kustomize plugin rules.
.PHONY: lint-kustomize
lint-kustomize: install-tools $(builtinplugins)
lint-kustomize: $(MYGOBIN)/golangci-lint-kustomize $(builtinplugins)
cd api; $(MYGOBIN)/golangci-lint-kustomize \
-c ../.golangci-kustomize.yml \
run ./...
@@ -251,10 +243,10 @@ test-unit-kustomize-all: \
test-unit-kustomize-plugins
test-unit-cmd-all:
./scripts/kyaml-pre-commit.sh
./hack/kyaml-pre-commit.sh
test-go-mod:
./scripts/check-go-mod.sh
./hack/check-go-mod.sh
# Environment variables are defined at
# https://github.com/kubernetes/test-infra/blob/master/prow/jobs.md#job-environment-variables
@@ -266,7 +258,7 @@ test-multi-module: $(MYGOBIN)/prchecker
export REPO_NAME=$(REPO_NAME); \
export PULL_NUMBER=$(PULL_NUMBER); \
export MODULES=$(MODULES); \
./scripts/check-multi-module.sh; \
./hack/check-multi-module.sh; \
)
.PHONY:
@@ -284,8 +276,8 @@ test-examples-kustomize-against-HEAD: $(MYGOBIN)/kustomize $(MYGOBIN)/mdrip
./hack/testExamplesAgainstKustomize.sh HEAD
.PHONY:
test-examples-kustomize-against-4.0: $(MYGOBIN)/mdrip
./hack/testExamplesAgainstKustomize.sh v4@v4.0.5
test-examples-kustomize-against-4.1: $(MYGOBIN)/mdrip
./hack/testExamplesAgainstKustomize.sh v4@v4.1.2
# linux only.
# This is for testing an example plugin that
@@ -358,8 +350,12 @@ $(MYGOBIN)/gh:
clean: clean-kustomize-external-go-plugin
go clean --cache
rm -f $(builtinplugins)
rm -f $(MYGOBIN)/kustomize
rm -f $(MYGOBIN)/goimports
rm -f $(MYGOBIN)/golangci-lint-kustomize
rm -f $(MYGOBIN)/kustomize
rm -f $(MYGOBIN)/mdrip
rm -f $(MYGOBIN)/prchecker
rm -f $(MYGOBIN)/stringer
# Handle pluginator manually.
# rm -f $(MYGOBIN)/pluginator

View File

@@ -27,16 +27,10 @@ func (p *AnnotationsTransformerPlugin) Transform(m resmap.ResMap) error {
if len(p.Annotations) == 0 {
return nil
}
for _, r := range m.Resources() {
err := r.ApplyFilter(annotations.Filter{
Annotations: p.Annotations,
FsSlice: p.FieldSpecs,
})
if err != nil {
return err
}
}
return nil
return m.ApplyFilter(annotations.Filter{
Annotations: p.Annotations,
FsSlice: p.FieldSpecs,
})
}
func NewAnnotationsTransformerPlugin() resmap.TransformerPlugin {

View File

@@ -267,6 +267,9 @@ func (p *HelmChartInflationGeneratorPlugin) templateCommand() []string {
if p.ReleaseName != "" {
args = append(args, p.ReleaseName)
}
if p.Namespace != "" {
args = append(args, "--namespace", p.Namespace)
}
args = append(args, filepath.Join(p.absChartHome(), p.Name))
if p.ValuesFile != "" {
args = append(args, "--values", p.ValuesFile)
@@ -277,6 +280,9 @@ func (p *HelmChartInflationGeneratorPlugin) templateCommand() []string {
// I've tried placing the flag before and after the name argument.
args = append(args, "--generate-name")
}
if p.IncludeCRDs {
args = append(args, "--include-crds")
}
return args
}

View File

@@ -0,0 +1,33 @@
// Code generated by pluginator on IAMPolicyGenerator; DO NOT EDIT.
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins
import (
"sigs.k8s.io/kustomize/api/filters/iampolicygenerator"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/yaml"
)
type IAMPolicyGeneratorPlugin struct {
types.IAMPolicyGeneratorArgs
}
func (p *IAMPolicyGeneratorPlugin) Config(h *resmap.PluginHelpers, config []byte) (err error) {
p.IAMPolicyGeneratorArgs = types.IAMPolicyGeneratorArgs{}
err = yaml.Unmarshal(config, p)
return
}
func (p *IAMPolicyGeneratorPlugin) Generate() (resmap.ResMap, error) {
r := resmap.New()
err := r.ApplyFilter(iampolicygenerator.Filter{
IAMPolicyGenerator: p.IAMPolicyGeneratorArgs,
})
return r, err
}
func NewIAMPolicyGeneratorPlugin() resmap.GeneratorPlugin {
return &IAMPolicyGeneratorPlugin{}
}

View File

@@ -25,24 +25,15 @@ func (p *ImageTagTransformerPlugin) Config(
}
func (p *ImageTagTransformerPlugin) Transform(m resmap.ResMap) error {
for _, r := range m.Resources() {
// traverse all fields at first
err := r.ApplyFilter(imagetag.LegacyFilter{
ImageTag: p.ImageTag,
})
if err != nil {
return err
}
// then use user specified field specs
err = r.ApplyFilter(imagetag.Filter{
ImageTag: p.ImageTag,
FsSlice: p.FieldSpecs,
})
if err != nil {
return err
}
if err := m.ApplyFilter(imagetag.LegacyFilter{
ImageTag: p.ImageTag,
}); err != nil {
return err
}
return nil
return m.ApplyFilter(imagetag.Filter{
ImageTag: p.ImageTag,
FsSlice: p.FieldSpecs,
})
}
func NewImageTagTransformerPlugin() resmap.TransformerPlugin {

View File

@@ -27,16 +27,10 @@ func (p *LabelTransformerPlugin) Transform(m resmap.ResMap) error {
if len(p.Labels) == 0 {
return nil
}
for _, r := range m.Resources() {
err := r.ApplyFilter(labels.Filter{
Labels: p.Labels,
FsSlice: p.FieldSpecs,
})
if err != nil {
return err
}
}
return nil
return m.ApplyFilter(labels.Filter{
Labels: p.Labels,
FsSlice: p.FieldSpecs,
})
}
func NewLabelTransformerPlugin() resmap.TransformerPlugin {

View File

@@ -30,7 +30,7 @@ func (p *NamespaceTransformerPlugin) Transform(m resmap.ResMap) error {
return nil
}
for _, r := range m.Resources() {
if r.IsEmpty() {
if r.IsNilOrEmpty() {
// Don't mutate empty objects?
continue
}

View File

@@ -62,10 +62,10 @@ func (p *PatchTransformerPlugin) Config(
if errSM == nil {
p.loadedPatch = patchSM
if p.Options["allowNameChange"] {
p.loadedPatch.SetAllowNameChange("true")
p.loadedPatch.AllowNameChange()
}
if p.Options["allowKindChange"] {
p.loadedPatch.SetAllowKindChange("true")
p.loadedPatch.AllowKindChange()
}
} else {
p.decodedPatch = patchJson

View File

@@ -7,9 +7,9 @@ import (
"errors"
"sigs.k8s.io/kustomize/api/filters/prefixsuffix"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/yaml"
)
@@ -73,12 +73,11 @@ func (p *PrefixSuffixTransformerPlugin) Transform(m resmap.ResMap) error {
r.StorePreviousId()
}
}
err := r.ApplyFilter(prefixsuffix.Filter{
if err := r.ApplyFilter(prefixsuffix.Filter{
Prefix: p.Prefix,
Suffix: p.Suffix,
FieldSpec: fs,
})
if err != nil {
}); err != nil {
return err
}
}

View File

@@ -9,7 +9,6 @@ import (
"sigs.k8s.io/kustomize/api/filters/replacement"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
kyaml "sigs.k8s.io/kustomize/kyaml/yaml"
"sigs.k8s.io/yaml"
)
@@ -50,14 +49,9 @@ func (p *ReplacementTransformerPlugin) Config(
}
func (p *ReplacementTransformerPlugin) Transform(m resmap.ResMap) (err error) {
var nodes []*kyaml.RNode
for _, r := range m.Resources() {
nodes = append(nodes, r.Node())
}
_, err = replacement.Filter{
return m.ApplyFilter(replacement.Filter{
Replacements: p.Replacements,
}.Filter(nodes)
return err
})
}
func NewReplacementTransformerPlugin() resmap.TransformerPlugin {

View File

@@ -7,9 +7,9 @@ import (
"fmt"
"sigs.k8s.io/kustomize/api/filters/replicacount"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/yaml"
)

7
api/filesys/doc.go Normal file
View File

@@ -0,0 +1,7 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package filesys provides a file system abstraction,
// a subset of that provided by golang.org/pkg/os,
// with an on-disk and in-memory representation.
package filesys

View File

@@ -1,7 +1,6 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package filesys provides a file system abstraction layer.
package filesys
import (
@@ -29,6 +28,8 @@ type FileSystem interface {
Open(path string) (File, error)
// IsDir returns true if the path is a directory.
IsDir(path string) bool
// ReadDir returns a list of files and directories within a directory.
ReadDir(path string) ([]string, error)
// CleanedAbs converts the given path into a
// directory and a file name, where the directory
// is represented as a ConfirmedDir and all that implies.

View File

@@ -349,6 +349,29 @@ func (n *fsNode) IsDir(path string) bool {
return result.isNodeADir()
}
// ReadDir implements FileSystem.
func (n *fsNode) ReadDir(path string) ([]string, error) {
if !n.IsDir(path) {
return nil, fmt.Errorf("%s is not a directory", path)
}
dir, err := n.Find(path)
if err != nil {
return nil, err
}
if dir == nil {
return nil, fmt.Errorf("could not find directory %s", path)
}
keys := make([]string, len(dir.dir))
i := 0
for k := range dir.dir {
keys[i] = k
i++
}
return keys, nil
}
// Size returns the size of the node.
func (n *fsNode) Size() int64 {
if n.isNodeADir() {
@@ -560,7 +583,7 @@ func isLegalFileNameForCreation(n string) bool {
func (n *fsNode) RegExpGlob(pattern string) ([]string, error) {
var result []string
var expression = regexp.MustCompile(pattern)
n.WalkMe(func(path string, info os.FileInfo, err error) error {
err := n.WalkMe(func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
@@ -571,6 +594,9 @@ func (n *fsNode) RegExpGlob(pattern string) ([]string, error) {
}
return nil
})
if err != nil {
return nil, err
}
sort.Strings(result)
return result, nil
}
@@ -582,7 +608,7 @@ func (n *fsNode) RegExpGlob(pattern string) ([]string, error) {
// This is how /bin/ls behaves.
func (n *fsNode) Glob(pattern string) ([]string, error) {
var result []string
n.WalkMe(func(path string, info os.FileInfo, err error) error {
err := n.WalkMe(func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
@@ -597,6 +623,9 @@ func (n *fsNode) Glob(pattern string) ([]string, error) {
}
return nil
})
if err != nil {
return nil, err
}
sort.Strings(result)
return result, nil
}

View File

@@ -100,6 +100,19 @@ func (fsOnDisk) IsDir(name string) bool {
return info.IsDir()
}
// ReadDir delegates to os.ReadDir
func (fsOnDisk) ReadDir(name string) ([]string, error) {
dirEntries, err := os.ReadDir(name)
if err != nil {
return nil, err
}
result := make([]string, len(dirEntries))
for i := range dirEntries {
result[i] = dirEntries[i].Name()
}
return result, nil
}
// ReadFile delegates to ioutil.ReadFile.
func (fsOnDisk) ReadFile(name string) ([]byte, error) { return ioutil.ReadFile(name) }

5
api/filters/doc.go Normal file
View File

@@ -0,0 +1,5 @@
package filters
// Package filters collects various implementations
// sigs.k8s.io/kustomize/kyaml/kio.Filter used by kustomize
// transformers to modify kubernetes objects.

View File

@@ -11,6 +11,7 @@ import (
"sigs.k8s.io/kustomize/api/internal/utils"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/errors"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
@@ -45,12 +46,11 @@ type Filter struct {
func (fltr Filter) Filter(obj *yaml.RNode) (*yaml.RNode, error) {
// check if the FieldSpec applies to the object
if match, err := isMatchGVK(fltr.FieldSpec, obj); !match || err != nil {
return obj, errors.Wrap(err)
if match := isMatchGVK(fltr.FieldSpec, obj); !match {
return obj, nil
}
fltr.path = utils.PathSplitter(fltr.FieldSpec.Path)
err := fltr.filter(obj)
if err != nil {
fltr.path = utils.PathSplitter(fltr.FieldSpec.Path, "/")
if err := fltr.filter(obj); err != nil {
s, _ := obj.String()
return nil, errors.WrapPrefixf(err,
"considering field '%s' of object\n%v", fltr.FieldSpec.Path, s)
@@ -158,28 +158,24 @@ func isSequenceField(name string) (string, bool) {
}
// isMatchGVK returns true if the fs.GVK matches the obj GVK.
func isMatchGVK(fs types.FieldSpec, obj *yaml.RNode) (bool, error) {
meta, err := obj.GetMeta()
if err != nil {
return false, err
}
if fs.Kind != "" && fs.Kind != meta.Kind {
func isMatchGVK(fs types.FieldSpec, obj *yaml.RNode) bool {
if kind := obj.GetKind(); fs.Kind != "" && fs.Kind != kind {
// kind doesn't match
return false, err
return false
}
// parse the group and version from the apiVersion field
group, version := parseGV(meta.APIVersion)
group, version := resid.ParseGroupVersion(obj.GetApiVersion())
if fs.Group != "" && fs.Group != group {
// group doesn't match
return false, nil
return false
}
if fs.Version != "" && fs.Version != version {
// version doesn't match
return false, nil
return false
}
return true, nil
return true
}

View File

@@ -46,10 +46,11 @@ xxx:
"empty path": {
fieldSpec: `
group: foo
version: v1
kind: Bar
`,
input: `
apiVersion: foo
apiVersion: foo/v1
kind: Bar
xxx:
`,
@@ -59,7 +60,7 @@ kind: Bar
xxx:
`,
error: `considering field '' of object
apiVersion: foo
apiVersion: foo/v1
kind: Bar
xxx:
: cannot set or create an empty field name`,
@@ -195,11 +196,14 @@ kind: Bar
input: `
a:
b: c
`,
expected: `
a:
b: c
`,
filter: fieldspec.Filter{
SetValue: filtersutil.SetScalar("e"),
},
error: "missing Resource metadata",
},
"miss-match-type": {

View File

@@ -1,49 +0,0 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package fieldspec
import (
"strings"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
// Return true for 'v' followed by a 1 or 2, and don't look at rest.
// I.e. 'v1', 'v1beta1', 'v2', would return true.
func looksLikeACoreApiVersion(s string) bool {
if len(s) < 2 {
return false
}
if s[0:1] != "v" {
return false
}
return s[1:2] == "1" || s[1:2] == "2"
}
// parseGV parses apiVersion field into group and version.
func parseGV(apiVersion string) (group, version string) {
// parse the group and version from the apiVersion field
parts := strings.SplitN(apiVersion, "/", 2)
group = parts[0]
if len(parts) > 1 {
version = parts[1]
}
// Special case the original "apiVersion" of what
// we now call the "core" (empty) group.
if version == "" && looksLikeACoreApiVersion(group) {
version = group
group = ""
}
return
}
// GetGVK parses the metadata into a GVK
func GetGVK(meta yaml.ResourceMeta) resid.Gvk {
group, version := parseGV(meta.APIVersion)
return resid.Gvk{
Group: group,
Version: version,
Kind: meta.Kind,
}
}

View File

@@ -1,156 +0,0 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package fieldspec
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
func TestParseGV(t *testing.T) {
testCases := map[string]struct {
input string
expectedGroup string
expectedVersion string
}{
"empty": {
input: "",
expectedGroup: "",
expectedVersion: "",
},
"certSigning": {
input: "certificates.k8s.io/v1beta1",
expectedGroup: "certificates.k8s.io",
expectedVersion: "v1beta1",
},
"extensions": {
input: "extensions/v1beta1",
expectedGroup: "extensions",
expectedVersion: "v1beta1",
},
"normal": {
input: "apps/v1",
expectedGroup: "apps",
expectedVersion: "v1",
},
"justApps": {
input: "apps",
expectedGroup: "apps",
expectedVersion: "",
},
"coreV1": {
input: "v1",
expectedGroup: "",
expectedVersion: "v1",
},
"coreV2": {
input: "v2",
expectedGroup: "",
expectedVersion: "v2",
},
"coreV2Beta1": {
input: "v2beta1",
expectedGroup: "",
expectedVersion: "v2beta1",
},
}
for tn, tc := range testCases {
t.Run(tn, func(t *testing.T) {
group, version := parseGV(tc.input)
if !assert.Equal(t, tc.expectedGroup, group) {
t.FailNow()
}
if !assert.Equal(t, tc.expectedVersion, version) {
t.FailNow()
}
})
}
}
func TestGetGVK(t *testing.T) {
testCases := map[string]struct {
input string
expected resid.Gvk
parseError string
metaError string
}{
"empty": {
input: `
`,
parseError: "EOF",
},
"junk": {
input: `
congress: effective
`,
metaError: "missing Resource metadata",
},
"normal": {
input: `
apiVersion: apps/v1
kind: Deployment
`,
expected: resid.Gvk{Group: "apps", Version: "v1", Kind: "Deployment"},
},
"apiVersionOnlyWithSlash": {
input: `
apiVersion: apps/v1
`,
expected: resid.Gvk{Group: "apps", Version: "v1", Kind: ""},
},
"apiVersionOnlyNoSlash1": {
input: `
apiVersion: apps
`,
expected: resid.Gvk{Group: "apps", Version: "", Kind: ""},
},
"apiVersionOnlyNoSlash2": {
input: `
apiVersion: v1
`,
expected: resid.Gvk{Group: "", Version: "v1", Kind: ""},
},
}
for tn, tc := range testCases {
t.Run(tn, func(t *testing.T) {
obj, err := yaml.Parse(tc.input)
if len(tc.parseError) != 0 {
if err == nil {
t.Error("expected parse error")
return
}
if !strings.Contains(err.Error(), tc.parseError) {
t.Errorf("expected parse err '%s', got '%v'", tc.parseError, err)
}
return
}
if !assert.NoError(t, err) {
t.FailNow()
}
meta, err := obj.GetMeta()
if len(tc.metaError) != 0 {
if err == nil {
t.Error("expected meta error")
return
}
if !strings.Contains(err.Error(), tc.metaError) {
t.Errorf("expected meta err '%s', got '%v'", tc.metaError, err)
}
return
}
if !assert.NoError(t, err) {
t.FailNow()
}
gvk := GetGVK(meta)
if !assert.Equal(t, tc.expected, gvk) {
t.FailNow()
}
})
}
}

View File

@@ -0,0 +1,3 @@
// Package gkesagenerator contains a kio.Filter that that generates a
// iampolicy-related resources for a given cloud provider
package iampolicygenerator

View File

@@ -0,0 +1,46 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package iampolicygenerator
import (
"log"
"os"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
func ExampleFilter() {
f := Filter{}
var err = yaml.Unmarshal([]byte(`
cloud: gke
kubernetesService:
namespace: k8s-namespace
name: k8s-sa-name
serviceAccount:
name: gsa-name
projectId: project-id
`), &f)
if err != nil {
log.Fatal(err)
}
err = kio.Pipeline{
Inputs: []kio.Reader{},
Filters: []kio.Filter{f},
Outputs: []kio.Writer{kio.ByteWriter{Writer: os.Stdout}},
}.Execute()
if err != nil {
log.Fatal(err)
}
// Output:
// apiVersion: v1
// kind: ServiceAccount
// metadata:
// annotations:
// iam.gke.io/gcp-service-account: gsa-name@project-id.iam.gserviceaccount.com
// name: k8s-sa-name
// namespace: k8s-namespace
}

View File

@@ -0,0 +1,55 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package iampolicygenerator
import (
"fmt"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
type Filter struct {
IAMPolicyGenerator types.IAMPolicyGeneratorArgs `json:",inline,omitempty" yaml:",inline,omitempty"`
}
// Filter adds a GKE service account object to nodes
func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
switch f.IAMPolicyGenerator.Cloud {
case types.GKE:
IAMPolicyResources, err := f.generateGkeIAMPolicyResources()
if err != nil {
return nil, err
}
nodes = append(nodes, IAMPolicyResources...)
default:
return nil, fmt.Errorf("cloud provider %s not supported yet", f.IAMPolicyGenerator.Cloud)
}
return nodes, nil
}
func (f Filter) generateGkeIAMPolicyResources() ([]*yaml.RNode, error) {
var result []*yaml.RNode
input := fmt.Sprintf(`
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
iam.gke.io/gcp-service-account: %s@%s.iam.gserviceaccount.com
name: %s
`, f.IAMPolicyGenerator.ServiceAccount.Name,
f.IAMPolicyGenerator.ProjectId,
f.IAMPolicyGenerator.KubernetesService.Name)
if f.IAMPolicyGenerator.Namespace != "" {
input = input + fmt.Sprintf("\n namespace: %s", f.IAMPolicyGenerator.Namespace)
}
sa, err := yaml.Parse(input)
if err != nil {
return nil, err
}
return append(result, sa), nil
}

View File

@@ -0,0 +1,75 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package iampolicygenerator
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
filtertest "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
)
func TestFilter(t *testing.T) {
testCases := map[string]struct {
args types.IAMPolicyGeneratorArgs
expected string
}{
"with namespace": {
args: types.IAMPolicyGeneratorArgs{
Cloud: types.GKE,
KubernetesService: types.KubernetesService{
Namespace: "k8s-namespace",
Name: "k8s-sa-name",
},
ServiceAccount: types.ServiceAccount{
Name: "gsa-name",
ProjectId: "project-id",
},
},
expected: `
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
iam.gke.io/gcp-service-account: gsa-name@project-id.iam.gserviceaccount.com
name: k8s-sa-name
namespace: k8s-namespace
`,
},
"without namespace": {
args: types.IAMPolicyGeneratorArgs{
Cloud: types.GKE,
KubernetesService: types.KubernetesService{
Name: "k8s-sa-name",
},
ServiceAccount: types.ServiceAccount{
Name: "gsa-name",
ProjectId: "project-id",
},
},
expected: `
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
iam.gke.io/gcp-service-account: gsa-name@project-id.iam.gserviceaccount.com
name: k8s-sa-name
`,
},
}
for tn, tc := range testCases {
t.Run(tn, func(t *testing.T) {
f := Filter{
IAMPolicyGenerator: tc.args,
}
actual := filtertest.RunFilter(t, "", f)
if !assert.Equal(t, strings.TrimSpace(tc.expected), strings.TrimSpace(actual)) {
t.FailNow()
}
})
}
}

View File

@@ -8,9 +8,9 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"sigs.k8s.io/kustomize/api/resid"
filtertest_test "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
func TestLabels_Filter(t *testing.T) {

View File

@@ -6,11 +6,11 @@ import (
"github.com/pkg/errors"
"sigs.k8s.io/kustomize/api/filters/fieldspec"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/resource"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
@@ -184,7 +184,7 @@ func (f Filter) recordTheReferral(referral *resource.Resource) {
// getRoleRefGvk returns a Gvk in the roleRef field. Return error
// if the roleRef, roleRef/apiGroup or roleRef/kind is missing.
func getRoleRefGvk(n *yaml.RNode) (*resid.Gvk, error) {
func getRoleRefGvk(n *resource.Resource) (*resid.Gvk, error) {
roleRef, err := n.Pipe(yaml.Lookup("roleRef"))
if err != nil {
return nil, err
@@ -257,8 +257,7 @@ func previousIdSelectedByGvk(gvk *resid.Gvk) sieveFunc {
// If the we are updating a 'roleRef/name' field, the 'apiGroup' and 'kind'
// fields in the same 'roleRef' map must be considered.
// If either object is cluster-scoped (!IsNamespaceableKind), there
// can be a referral.
// If either object is cluster-scoped, there can be a referral.
// E.g. a RoleBinding (which exists in a namespace) can refer
// to a ClusterRole (cluster-scoped) object.
// https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole
@@ -270,7 +269,7 @@ func (f Filter) roleRefFilter() sieveFunc {
if !strings.HasSuffix(f.NameFieldToUpdate.Path, "roleRef/name") {
return acceptAll
}
roleRefGvk, err := getRoleRefGvk(f.Referrer.AsRNode())
roleRefGvk, err := getRoleRefGvk(f.Referrer)
if err != nil {
return acceptAll
}
@@ -285,12 +284,12 @@ func prefixSuffixEquals(other resource.ResCtx) sieveFunc {
func (f Filter) sameCurrentNamespaceAsReferrer() sieveFunc {
referrerCurId := f.Referrer.CurId()
if !referrerCurId.IsNamespaceableKind() {
if referrerCurId.IsClusterScoped() {
// If the referrer is cluster-scoped, let anything through.
return acceptAll
}
return func(r *resource.Resource) bool {
if !r.CurId().IsNamespaceableKind() {
if r.CurId().IsClusterScoped() {
// Allow cluster-scoped through.
return true
}

View File

@@ -6,10 +6,10 @@ import (
"github.com/stretchr/testify/assert"
"sigs.k8s.io/kustomize/api/provider"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/resmap"
filtertest_test "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
func TestNamerefFilter(t *testing.T) {

View File

@@ -4,11 +4,11 @@
package namespace
import (
"sigs.k8s.io/kustomize/api/filters/fieldspec"
"sigs.k8s.io/kustomize/api/filters/filtersutil"
"sigs.k8s.io/kustomize/api/filters/fsslice"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
@@ -54,16 +54,11 @@ func (ns Filter) run(node *yaml.RNode) (*yaml.RNode, error) {
// hacks applies the namespace transforms that are hardcoded rather
// than specified through FieldSpecs.
func (ns Filter) hacks(obj *yaml.RNode) error {
meta, err := obj.GetMeta()
if err != nil {
gvk := resid.GvkFromNode(obj)
if err := ns.metaNamespaceHack(obj, gvk); err != nil {
return err
}
if err := ns.metaNamespaceHack(obj, meta); err != nil {
return err
}
return ns.roleBindingHack(obj, meta)
return ns.roleBindingHack(obj, gvk)
}
// metaNamespaceHack is a hack for implementing the namespace transform
@@ -74,9 +69,8 @@ func (ns Filter) hacks(obj *yaml.RNode) error {
// This hack should be updated to allow individual resources to specify
// if they are cluster scoped through either an annotation on the resources,
// or through inlined OpenAPI on the resource as a YAML comment.
func (ns Filter) metaNamespaceHack(obj *yaml.RNode, meta yaml.ResourceMeta) error {
gvk := fieldspec.GetGVK(meta)
if !gvk.IsNamespaceableKind() {
func (ns Filter) metaNamespaceHack(obj *yaml.RNode, gvk resid.Gvk) error {
if gvk.IsClusterScoped() {
return nil
}
f := fsslice.Filter{
@@ -104,8 +98,8 @@ func (ns Filter) metaNamespaceHack(obj *yaml.RNode, meta yaml.ResourceMeta) erro
// ...
// - name: "something-else" # this will not have the namespace set
// ...
func (ns Filter) roleBindingHack(obj *yaml.RNode, meta yaml.ResourceMeta) error {
if meta.Kind != roleBindingKind && meta.Kind != clusterRoleBindingKind {
func (ns Filter) roleBindingHack(obj *yaml.RNode, gvk resid.Gvk) error {
if gvk.Kind != roleBindingKind && gvk.Kind != clusterRoleBindingKind {
return nil
}

View File

@@ -7,8 +7,9 @@ import (
"fmt"
"strings"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/internal/utils"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
@@ -66,7 +67,7 @@ func rejectId(rejects []*types.Selector, id *resid.ResId) bool {
func applyToNode(node *yaml.RNode, value *yaml.RNode, target *types.TargetSelector) error {
for _, fp := range target.FieldPaths {
fieldPath := strings.Split(fp, ".")
fieldPath := utils.SmarterPathSplitter(fp, ".")
var t *yaml.RNode
var err error
if target.Options != nil && target.Options.Create {
@@ -87,12 +88,11 @@ func applyToNode(node *yaml.RNode, value *yaml.RNode, target *types.TargetSelect
}
func setTargetValue(options *types.FieldOptions, t *yaml.RNode, value *yaml.RNode) error {
value = value.Copy()
if options != nil && options.Delimiter != "" {
if t.YNode().Kind != yaml.ScalarNode {
return fmt.Errorf("delimiter option can only be used with scalar nodes")
}
tv := strings.Split(t.YNode().Value, options.Delimiter)
v := yaml.GetValue(value)
// TODO: Add a way to remove an element
@@ -119,7 +119,7 @@ func getReplacement(nodes []*yaml.RNode, r *types.Replacement) (*yaml.RNode, err
if r.Source.FieldPath == "" {
r.Source.FieldPath = types.DefaultReplacementFieldPath
}
fieldPath := strings.Split(r.Source.FieldPath, ".")
fieldPath := utils.SmarterPathSplitter(r.Source.FieldPath, ".")
rn, err := source.Pipe(yaml.Lookup(fieldPath...))
if err != nil {
@@ -168,11 +168,6 @@ func selectSourceNode(nodes []*yaml.RNode, selector *types.SourceSelector) (*yam
// makeResId makes a ResId from an RNode.
func makeResId(n *yaml.RNode) *resid.ResId {
ns, err := n.GetNamespace()
if err != nil {
// Resource has no metadata (no apiVersion, kind, nor metadata field).
return nil
}
apiVersion := n.Field(yaml.APIVersionField)
var group, version string
if apiVersion != nil {
@@ -181,6 +176,6 @@ func makeResId(n *yaml.RNode) *resid.ResId {
return &resid.ResId{
Gvk: resid.Gvk{Group: group, Version: version, Kind: n.GetKind()},
Name: n.GetName(),
Namespace: ns,
Namespace: n.GetNamespace(),
}
}

View File

@@ -1338,6 +1338,148 @@ spec:
`,
expectedErr: "delimiter option can only be used with scalar nodes",
},
"list index contains '.' character": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: source
data:
value: example
---
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: some-secret
spec:
backendType: secretsManager
data:
- key: some-prefix-replaceme
name: .first
version: latest
property: first
- key: some-prefix-replaceme
name: second
version: latest
property: second
`,
replacements: `replacements:
- source:
kind: ConfigMap
version: v1
name: source
fieldPath: data.value
targets:
- select:
group: kubernetes-client.io
version: v1
kind: ExternalSecret
name: some-secret
fieldPaths:
- spec.data.[name=.first].key
- spec.data.[name=second].key
options:
delimiter: "-"
index: 2
`,
expected: `apiVersion: v1
kind: ConfigMap
metadata:
name: source
data:
value: example
---
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: some-secret
spec:
backendType: secretsManager
data:
- key: some-prefix-example
name: .first
version: latest
property: first
- key: some-prefix-example
name: second
version: latest
property: second`,
},
"multiple field paths in target": {
input: `apiVersion: v1
kind: ConfigMap
metadata:
name: source
data:
value: example
---
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: some-secret
spec:
backendType: secretsManager
data:
- key: some-prefix-replaceme
name: first
version: latest
property: first
- key: some-prefix-replaceme
name: second
version: latest
property: second
- key: some-prefix-replaceme
name: third
version: latest
property: third
`,
replacements: `replacements:
- source:
kind: ConfigMap
version: v1
name: source
fieldPath: data.value
targets:
- select:
group: kubernetes-client.io
version: v1
kind: ExternalSecret
name: some-secret
fieldPaths:
- spec.data.0.key
- spec.data.1.key
- spec.data.2.key
options:
delimiter: "-"
index: 2
`,
expected: `apiVersion: v1
kind: ConfigMap
metadata:
name: source
data:
value: example
---
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: some-secret
spec:
backendType: secretsManager
data:
- key: some-prefix-example
name: first
version: latest
property: first
- key: some-prefix-example
name: second
version: latest
property: second
- key: some-prefix-example
name: third
version: latest
property: third
`,
},
}
for tn, tc := range testCases {
@@ -1353,7 +1495,7 @@ spec:
t.Errorf("unexpected error: %s\n", err.Error())
t.FailNow()
}
if !assert.Equal(t, tc.expectedErr, err.Error()) {
if !assert.Contains(t, err.Error(), tc.expectedErr) {
t.FailNow()
}
}

View File

@@ -11,8 +11,10 @@ require (
github.com/stretchr/testify v1.5.1
gopkg.in/yaml.v2 v2.4.0
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e
sigs.k8s.io/kustomize/kyaml v0.10.18
sigs.k8s.io/kustomize/kyaml v0.10.20
sigs.k8s.io/yaml v1.2.0
)
replace gopkg.in/yaml.v3 => gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
replace sigs.k8s.io/kustomize/kyaml => ../kyaml

View File

@@ -47,7 +47,6 @@ github.com/go-openapi/jsonreference v0.19.3/go.mod h1:rjx6GuL8TTa9VaixXglHmQmIL9
github.com/go-openapi/swag v0.19.5 h1:lTz6Ys4CmqqCQmZPBlbQENR1/GucA2bzYTE12Pw4tFY=
github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/gobuffalo/here v0.6.0/go.mod h1:wAG085dHOYqUpf+Ap+WOdrPTp5IYcDAs/x7PLa8Y5fM=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
@@ -98,7 +97,6 @@ github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN
github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mailru/easyjson v0.7.0 h1:aizVhC/NAAcKWb+5QsU1iNOZb4Yws5UO2I+aIprQITM=
github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs=
github.com/markbates/pkger v0.17.1/go.mod h1:0JoVlrol20BSywW79rN3kdFFsE5xYM+rSCQDXbLhiuI=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
@@ -216,7 +214,6 @@ gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bl
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.7/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
@@ -227,8 +224,6 @@ k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8
k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE=
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e h1:KLHHjkdQFomZy8+06csTWZ0m1343QqxZhR2LJ1OxCYM=
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e/go.mod h1:vHXdDvt9+2spS2Rx9ql3I8tycm3H9FDfdUoIuKCefvw=
sigs.k8s.io/kustomize/kyaml v0.10.18 h1:Cuf4KiVULTttfo/2Vls2H9fA7eH8Xll1w6RgGdL+tR8=
sigs.k8s.io/kustomize/kyaml v0.10.18/go.mod h1:h94DSoDbmnN4BTc6VTX7tGNGXZy29rbPo+R4jGMvA8U=
sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw=
sigs.k8s.io/yaml v1.2.0 h1:kr/MCeFWJWTwyaHoR9c8EjH9OumOmoF9YGiZd7lFm/Q=
sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc=

View File

@@ -12,8 +12,8 @@ import (
"sigs.k8s.io/kustomize/api/filesys"
"sigs.k8s.io/kustomize/api/ifc"
"sigs.k8s.io/kustomize/api/internal/plugins/builtinconfig"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/yaml"
)
@@ -178,9 +178,12 @@ func loadCrdIntoConfig(
}
}
if property.Ref.GetURL() != nil {
loadCrdIntoConfig(
err = loadCrdIntoConfig(
theConfig, theGvk, theMap,
property.Ref.String(), append(path, propName))
if err != nil {
return
}
}
}
return nil

View File

@@ -7,12 +7,13 @@ import (
"reflect"
"testing"
"github.com/stretchr/testify/require"
"sigs.k8s.io/kustomize/api/filesys"
. "sigs.k8s.io/kustomize/api/internal/accumulator"
"sigs.k8s.io/kustomize/api/internal/plugins/builtinconfig"
"sigs.k8s.io/kustomize/api/loader"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
// This defines two CRD's: Bee and MyKind.
@@ -162,16 +163,13 @@ func TestLoadCRDs(t *testing.T) {
}
fSys := filesys.MakeFsInMemory()
fSys.WriteFile("/testpath/crd.json", []byte(crdContent))
err := fSys.WriteFile("/testpath/crd.json", []byte(crdContent))
require.NoError(t, err)
ldr, err := loader.NewLoader(loader.RestrictionRootOnly, "/testpath", fSys)
if err != nil {
t.Fatalf("unexpected error:%v", err)
}
require.NoError(t, err)
actualTc, err := LoadConfigFromCRDs(ldr, []string{"crd.json"})
if err != nil {
t.Fatalf("unexpected error:%v", err)
}
require.NoError(t, err)
if !reflect.DeepEqual(actualTc, expectedTc) {
t.Fatalf("expected\n %v\n but got\n %v\n", expectedTc, actualTc)
}

View File

@@ -9,9 +9,9 @@ import (
"sigs.k8s.io/kustomize/api/internal/plugins/builtinconfig"
"sigs.k8s.io/kustomize/api/provider"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/resmap"
resmaptest_test "sigs.k8s.io/kustomize/api/testutils/resmaptest"
"sigs.k8s.io/kustomize/kyaml/resid"
)
const notEqualErrFmt = "expected (self) doesn't match actual (other): %v"
@@ -885,9 +885,9 @@ func TestNameReferenceClusterWide(t *testing.T) {
}).ResMap()
clusterRoleId := resid.NewResId(
resid.Gvk{Group: "rbac.authorization.k8s.io", Version: "v1", Kind: "ClusterRole"}, modifiedname)
resid.NewGvk("rbac.authorization.k8s.io", "v1", "ClusterRole"), modifiedname)
clusterRoleBindingId := resid.NewResId(
resid.Gvk{Group: "rbac.authorization.k8s.io", Version: "v1", Kind: "ClusterRoleBinding"}, modifiedname)
resid.NewGvk("rbac.authorization.k8s.io", "v1", "ClusterRoleBinding"), modifiedname)
clusterRole, _ := expected.GetByCurrentId(clusterRoleId)
clusterRole.AppendRefBy(clusterRoleBindingId)
@@ -1012,9 +1012,11 @@ func TestNameReferenceNamespaceTransformation(t *testing.T) {
}).ResMap()
clusterRoleId := resid.NewResId(
resid.Gvk{Group: "rbac.authorization.k8s.io", Version: "v1", Kind: "ClusterRole"}, modifiedname)
resid.NewGvk("rbac.authorization.k8s.io", "v1", "ClusterRole"),
modifiedname)
clusterRoleBindingId := resid.NewResId(
resid.Gvk{Group: "rbac.authorization.k8s.io", Version: "v1", Kind: "ClusterRoleBinding"}, modifiedname)
resid.NewGvk("rbac.authorization.k8s.io", "v1", "ClusterRoleBinding"),
modifiedname)
clusterRole, _ := expected.GetByCurrentId(clusterRoleId)
clusterRole.AppendRefBy(clusterRoleBindingId)

View File

@@ -7,10 +7,10 @@ import (
"reflect"
"testing"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/resmap"
resmaptest_test "sigs.k8s.io/kustomize/api/testutils/resmaptest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
func TestRefVarTransformer(t *testing.T) {

View File

@@ -9,9 +9,9 @@ import (
"strings"
"sigs.k8s.io/kustomize/api/internal/plugins/builtinconfig"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
// ResAccumulator accumulates resources and the rules
@@ -72,7 +72,7 @@ func (ra *ResAccumulator) MergeVars(incoming []types.Var) error {
for _, v := range incoming {
targetId := resid.NewResIdWithNamespace(v.ObjRef.GVK(), v.ObjRef.Name, v.ObjRef.Namespace)
idMatcher := targetId.GvknEquals
if targetId.Namespace != "" || !targetId.IsNamespaceableKind() {
if targetId.Namespace != "" || targetId.IsClusterScoped() {
// Preserve backward compatibility. An empty namespace means
// wildcard search on the namespace hence we still use GvknEquals
idMatcher = targetId.Equals
@@ -107,6 +107,7 @@ func (ra *ResAccumulator) findVarValueFromResources(v types.Var) (interface{}, e
for _, res := range ra.resMap.Resources() {
for _, varName := range res.GetRefVarNames() {
if varName == v.Name {
//nolint: staticcheck
s, err := res.GetFieldValue(v.FieldRef.FieldPath)
if err != nil {
return "", fmt.Errorf(

View File

@@ -10,14 +10,15 @@ import (
"strings"
"testing"
"github.com/stretchr/testify/require"
. "sigs.k8s.io/kustomize/api/internal/accumulator"
"sigs.k8s.io/kustomize/api/internal/plugins/builtinconfig"
"sigs.k8s.io/kustomize/api/provider"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/resource"
resmaptest_test "sigs.k8s.io/kustomize/api/testutils/resmaptest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
func makeResAccumulator(t *testing.T) *ResAccumulator {
@@ -224,20 +225,26 @@ func TestResolveVarConflicts(t *testing.T) {
// create accumulators holding apparently conflicting vars that are not
// actually in conflict because they point to the same concrete value.
rm0 := resmap.New()
rm0.Append(rf.FromMap(fooAws))
err := rm0.Append(rf.FromMap(fooAws))
require.NoError(t, err)
ac0 := MakeEmptyAccumulator()
ac0.AppendAll(rm0)
ac0.MergeVars([]types.Var{varFoo})
err = ac0.AppendAll(rm0)
require.NoError(t, err)
err = ac0.MergeVars([]types.Var{varFoo})
require.NoError(t, err)
rm1 := resmap.New()
rm1.Append(rf.FromMap(barAws))
err = rm1.Append(rf.FromMap(barAws))
require.NoError(t, err)
ac1 := MakeEmptyAccumulator()
ac1.AppendAll(rm1)
ac1.MergeVars([]types.Var{varBar})
err = ac1.AppendAll(rm1)
require.NoError(t, err)
err = ac1.MergeVars([]types.Var{varBar})
require.NoError(t, err)
// validate that two vars of the same name which reference the same concrete
// value do not produce a conflict.
err := ac0.MergeAccumulator(ac1)
err = ac0.MergeAccumulator(ac1)
if err == nil {
t.Fatalf("see bug gh-1600")
}
@@ -246,10 +253,13 @@ func TestResolveVarConflicts(t *testing.T) {
// two above (because it contains a variable whose name is used in the other
// accumulators AND whose concrete values are different).
rm2 := resmap.New()
rm2.Append(rf.FromMap(barGcp))
err = rm2.Append(rf.FromMap(barGcp))
require.NoError(t, err)
ac2 := MakeEmptyAccumulator()
ac2.AppendAll(rm2)
ac2.MergeVars([]types.Var{varBar})
err = ac2.AppendAll(rm2)
require.NoError(t, err)
err = ac2.MergeVars([]types.Var{varBar})
require.NoError(t, err)
err = ac1.MergeAccumulator(ac2)
if err == nil {
t.Fatalf("dupe vars w/ different concrete values should conflict")

View File

@@ -40,7 +40,13 @@ func MakeConfigMap(
if err = rn.LoadMapIntoConfigMapData(m); err != nil {
return nil, err
}
copyLabelsAndAnnotations(rn, args.Options)
setImmutable(rn, args.Options)
err = copyLabelsAndAnnotations(rn, args.Options)
if err != nil {
return nil, err
}
err = setImmutable(rn, args.Options)
if err != nil {
return nil, err
}
return rn, nil
}

View File

@@ -14,7 +14,7 @@ type Cloner func(repoSpec *RepoSpec) error
// to say, some remote API, to obtain a local clone of
// a remote repo.
func ClonerUsingGitExec(repoSpec *RepoSpec) error {
r, err := newCmdRunner()
r, err := newCmdRunner(repoSpec.Timeout)
if err != nil {
return err
}
@@ -36,7 +36,10 @@ func ClonerUsingGitExec(repoSpec *RepoSpec) error {
if err = r.run("checkout", "FETCH_HEAD"); err != nil {
return err
}
return r.run("submodule", "update", "--init", "--recursive")
if repoSpec.Submodules {
return r.run("submodule", "update", "--init", "--recursive")
}
return nil
}
// DoNothingCloner returns a cloner that only sets

View File

@@ -12,9 +12,6 @@ import (
"sigs.k8s.io/kustomize/api/internal/utils"
)
// Arbitrary, but non-infinite, timeout for running commands.
const defaultDuration = 27 * time.Second
// gitRunner runs the external git binary.
type gitRunner struct {
gitProgram string
@@ -24,7 +21,7 @@ type gitRunner struct {
// newCmdRunner returns a gitRunner if it can find the binary.
// It also creats a temp directory for cloning repos.
func newCmdRunner() (*gitRunner, error) {
func newCmdRunner(timeout time.Duration) (*gitRunner, error) {
gitProgram, err := exec.LookPath("git")
if err != nil {
return nil, errors.Wrap(err, "no 'git' program on path")
@@ -35,7 +32,7 @@ func newCmdRunner() (*gitRunner, error) {
}
return &gitRunner{
gitProgram: gitProgram,
duration: defaultDuration,
duration: timeout,
dir: dir,
}, nil
}

View File

@@ -5,9 +5,11 @@ package git
import (
"fmt"
"net/url"
"path/filepath"
"regexp"
"strconv"
"strings"
"time"
"sigs.k8s.io/kustomize/api/filesys"
)
@@ -44,6 +46,12 @@ type RepoSpec struct {
// e.g. .git or empty in case of _git is present
GitSuffix string
// Submodules indicates whether or not to clone git submodules.
Submodules bool
// Timeout is the maximum duration allowed for execing git commands.
Timeout time.Duration
}
// CloneSpec returns a string suitable for "git clone {spec}".
@@ -70,6 +78,7 @@ func (x *RepoSpec) Cleaner(fSys filesys.FileSystem) func() error {
return func() error { return fSys.RemoveAll(x.Dir.String()) }
}
// NewRepoSpecFromUrl parses git-like urls.
// From strings like git@github.com:someOrg/someRepo.git or
// https://github.com/someOrg/someRepo?ref=someHash, extract
// the parts.
@@ -77,7 +86,7 @@ func NewRepoSpecFromUrl(n string) (*RepoSpec, error) {
if filepath.IsAbs(n) {
return nil, fmt.Errorf("uri looks like abs path: %s", n)
}
host, orgRepo, path, gitRef, gitSuffix := parseGitUrl(n)
host, orgRepo, path, gitRef, gitSubmodules, suffix, gitTimeout := parseGitUrl(n)
if orgRepo == "" {
return nil, fmt.Errorf("url lacks orgRepo: %s", n)
}
@@ -86,28 +95,28 @@ func NewRepoSpecFromUrl(n string) (*RepoSpec, error) {
}
return &RepoSpec{
raw: n, Host: host, OrgRepo: orgRepo,
Dir: notCloned, Path: path, Ref: gitRef, GitSuffix: gitSuffix}, nil
Dir: notCloned, Path: path, Ref: gitRef, GitSuffix: suffix,
Submodules: gitSubmodules, Timeout: gitTimeout}, nil
}
const (
refQuery = "?ref="
refQueryRegex = "\\?(version|ref)="
gitSuffix = ".git"
gitDelimiter = "_git/"
refQuery = "?ref="
gitSuffix = ".git"
gitDelimiter = "_git/"
)
// From strings like git@github.com:someOrg/someRepo.git or
// https://github.com/someOrg/someRepo?ref=someHash, extract
// the parts.
func parseGitUrl(n string) (
host string, orgRepo string, path string, gitRef string, gitSuff string) {
host string, orgRepo string, path string, gitRef string, gitSubmodules bool, gitSuff string, gitTimeout time.Duration) {
if strings.Contains(n, gitDelimiter) {
index := strings.Index(n, gitDelimiter)
// Adding _git/ to host
host = normalizeGitHostSpec(n[:index+len(gitDelimiter)])
orgRepo = strings.Split(strings.Split(n[index+len(gitDelimiter):], "/")[0], "?")[0]
path, gitRef = peelQuery(n[index+len(gitDelimiter)+len(orgRepo):])
path, gitRef, gitTimeout, gitSubmodules = peelQuery(n[index+len(gitDelimiter)+len(orgRepo):])
return
}
host, n = parseHostSpec(n)
@@ -116,35 +125,75 @@ func parseGitUrl(n string) (
index := strings.Index(n, gitSuffix)
orgRepo = n[0:index]
n = n[index+len(gitSuffix):]
path, gitRef = peelQuery(n)
if n[0] == '/' {
n = n[1:]
}
path, gitRef, gitTimeout, gitSubmodules = peelQuery(n)
return
}
i := strings.Index(n, "/")
if i < 1 {
return "", "", "", "", ""
path, gitRef, gitTimeout, gitSubmodules = peelQuery(n)
return
}
j := strings.Index(n[i+1:], "/")
if j >= 0 {
j += i + 1
orgRepo = n[:j]
path, gitRef = peelQuery(n[j+1:])
path, gitRef, gitTimeout, gitSubmodules = peelQuery(n[j+1:])
return
}
path = ""
orgRepo, gitRef = peelQuery(n)
return host, orgRepo, path, gitRef, gitSuff
orgRepo, gitRef, gitTimeout, gitSubmodules = peelQuery(n)
return host, orgRepo, path, gitRef, gitSubmodules, gitSuff, gitTimeout
}
func peelQuery(arg string) (string, string) {
// Clone git submodules by default.
const defaultSubmodules = true
r, _ := regexp.Compile(refQueryRegex)
j := r.FindStringIndex(arg)
// Arbitrary, but non-infinite, timeout for running commands.
const defaultTimeout = 27 * time.Second
if len(j) > 0 {
return arg[:j[0]], arg[j[0]+len(r.FindString(arg)):]
func peelQuery(arg string) (string, string, time.Duration, bool) {
// Parse the given arg into a URL. In the event of a parse failure, return
// our defaults.
parsed, err := url.Parse(arg)
if err != nil {
return arg, "", defaultTimeout, defaultSubmodules
}
return arg, ""
values := parsed.Query()
// ref is the desired git ref to target. Can be specified by in a git URL
// with ?ref=<string> or ?version=<string>, although ref takes precedence.
ref := values.Get("version")
if queryValue := values.Get("ref"); queryValue != "" {
ref = queryValue
}
// depth is the desired git exec timeout. Can be specified by in a git URL
// with ?timeout=<duration>.
duration := defaultTimeout
if queryValue := values.Get("timeout"); queryValue != "" {
// Attempt to first parse as a number of integer seconds (like "61"),
// and then attempt to parse as a suffixed duration (like "61s").
if intValue, err := strconv.Atoi(queryValue); err == nil && intValue > 0 {
duration = time.Duration(intValue) * time.Second
} else if durationValue, err := time.ParseDuration(queryValue); err == nil && durationValue > 0 {
duration = durationValue
}
}
// submodules indicates if git submodule cloning is desired. Can be
// specified by in a git URL with ?submodules=<bool>.
submodules := defaultSubmodules
if queryValue := values.Get("submodules"); queryValue != "" {
if boolValue, err := strconv.ParseBool(queryValue); err == nil {
submodules = boolValue
}
}
return parsed.Path, ref, duration, submodules
}
func parseHostSpec(n string) (string, string) {

View File

@@ -8,6 +8,9 @@ import (
"path/filepath"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
var orgRepos = []string{"someOrg/someRepo", "kubernetes/website"}
@@ -108,96 +111,87 @@ func TestNewRepoSpecFromUrlErrors(t *testing.T) {
}
func TestNewRepoSpecFromUrl_CloneSpecs(t *testing.T) {
testcases := []struct {
testcases := map[string]struct {
input string
cloneSpec string
absPath string
ref string
}{
{
"t1": {
input: "https://git-codecommit.us-east-2.amazonaws.com/someorg/somerepo/somedir",
cloneSpec: "https://git-codecommit.us-east-2.amazonaws.com/someorg/somerepo",
absPath: notCloned.Join("somedir"),
ref: "",
},
{
"t2": {
input: "https://git-codecommit.us-east-2.amazonaws.com/someorg/somerepo/somedir?ref=testbranch",
cloneSpec: "https://git-codecommit.us-east-2.amazonaws.com/someorg/somerepo",
absPath: notCloned.Join("somedir"),
ref: "testbranch",
},
{
"t3": {
input: "https://fabrikops2.visualstudio.com/someorg/somerepo?ref=master",
cloneSpec: "https://fabrikops2.visualstudio.com/someorg/somerepo",
absPath: notCloned.String(),
ref: "master",
},
{
"t4": {
input: "http://github.com/someorg/somerepo/somedir",
cloneSpec: "https://github.com/someorg/somerepo.git",
absPath: notCloned.Join("somedir"),
ref: "",
},
{
"t5": {
input: "git@github.com:someorg/somerepo/somedir",
cloneSpec: "git@github.com:someorg/somerepo.git",
absPath: notCloned.Join("somedir"),
ref: "",
},
{
"t6": {
input: "git@gitlab2.sqtools.ru:10022/infra/kubernetes/thanos-base.git?ref=v0.1.0",
cloneSpec: "git@gitlab2.sqtools.ru:10022/infra/kubernetes/thanos-base.git",
absPath: notCloned.String(),
ref: "v0.1.0",
},
{
"t7": {
input: "git@bitbucket.org:company/project.git//path?ref=branch",
cloneSpec: "git@bitbucket.org:company/project.git",
absPath: notCloned.Join("path"),
ref: "branch",
},
{
"t8": {
input: "https://itfs.mycompany.com/collection/project/_git/somerepos",
cloneSpec: "https://itfs.mycompany.com/collection/project/_git/somerepos",
absPath: notCloned.String(),
ref: "",
},
{
"t9": {
input: "https://itfs.mycompany.com/collection/project/_git/somerepos?version=v1.0.0",
cloneSpec: "https://itfs.mycompany.com/collection/project/_git/somerepos",
absPath: notCloned.String(),
ref: "v1.0.0",
},
{
"t10": {
input: "https://itfs.mycompany.com/collection/project/_git/somerepos/somedir?version=v1.0.0",
cloneSpec: "https://itfs.mycompany.com/collection/project/_git/somerepos",
absPath: notCloned.Join("somedir"),
ref: "v1.0.0",
},
{
"t11": {
input: "git::https://itfs.mycompany.com/collection/project/_git/somerepos",
cloneSpec: "https://itfs.mycompany.com/collection/project/_git/somerepos",
absPath: notCloned.String(),
ref: "",
},
}
for _, testcase := range testcases {
rs, err := NewRepoSpecFromUrl(testcase.input)
if err != nil {
t.Errorf("Unexpected error: %v", err)
}
if rs.CloneSpec() != testcase.cloneSpec {
t.Errorf("CloneSpec expected to be %v, but got %v on %s",
testcase.cloneSpec, rs.CloneSpec(), testcase.input)
}
if rs.AbsPath() != testcase.absPath {
t.Errorf("AbsPath expected to be %v, but got %v on %s",
testcase.absPath, rs.AbsPath(), testcase.input)
}
if rs.Ref != testcase.ref {
t.Errorf("ref expected to be %v, but got %v on %s",
testcase.ref, rs.Ref, testcase.input)
}
for tn, tc := range testcases {
t.Run(tn, func(t *testing.T) {
rs, err := NewRepoSpecFromUrl(tc.input)
assert.NoError(t, err)
assert.Equal(t, tc.cloneSpec, rs.CloneSpec(), "cloneSpec mismatch")
assert.Equal(t, tc.absPath, rs.AbsPath(), "absPath mismatch")
assert.Equal(t, tc.ref, rs.Ref, "ref mismatch")
})
}
}
@@ -232,28 +226,134 @@ func TestIsAzureHost(t *testing.T) {
}
func TestPeelQuery(t *testing.T) {
testcases := []struct {
input string
expect [2]string
testcases := map[string]struct {
input string
path string
ref string
submodules bool
timeout time.Duration
}{
{
input: "somerepos?ref=v1.0.0",
expect: [2]string{"somerepos", "v1.0.0"},
"t1": {
// All empty.
input: "somerepos",
path: "somerepos",
ref: "",
submodules: defaultSubmodules,
timeout: defaultTimeout,
},
{
input: "somerepos?version=master",
expect: [2]string{"somerepos", "master"},
"t2": {
input: "somerepos?ref=v1.0.0",
path: "somerepos",
ref: "v1.0.0",
submodules: defaultSubmodules,
timeout: defaultTimeout,
},
{
input: "somerepos",
expect: [2]string{"somerepos", ""},
"t3": {
input: "somerepos?version=master",
path: "somerepos",
ref: "master",
submodules: defaultSubmodules,
timeout: defaultTimeout,
},
"t4": {
// A ref value takes precedence over a version value.
input: "somerepos?version=master&ref=v1.0.0",
path: "somerepos",
ref: "v1.0.0",
submodules: defaultSubmodules,
timeout: defaultTimeout,
},
"t5": {
// Empty submodules value uses default.
input: "somerepos?version=master&submodules=",
path: "somerepos",
ref: "master",
submodules: defaultSubmodules,
timeout: defaultTimeout,
},
"t6": {
// Malformed submodules value uses default.
input: "somerepos?version=master&submodules=maybe",
path: "somerepos",
ref: "master",
submodules: defaultSubmodules,
timeout: defaultTimeout,
},
"t7": {
input: "somerepos?version=master&submodules=true",
path: "somerepos",
ref: "master",
submodules: true,
timeout: defaultTimeout,
},
"t8": {
input: "somerepos?version=master&submodules=false",
path: "somerepos",
ref: "master",
submodules: false,
timeout: defaultTimeout,
},
"t9": {
// Empty timeout value uses default.
input: "somerepos?version=master&timeout=",
path: "somerepos",
ref: "master",
submodules: defaultSubmodules,
timeout: defaultTimeout,
},
"t10": {
// Malformed timeout value uses default.
input: "somerepos?version=master&timeout=jiffy",
path: "somerepos",
ref: "master",
submodules: defaultSubmodules,
timeout: defaultTimeout,
},
"t11": {
// Zero timeout value uses default.
input: "somerepos?version=master&timeout=0",
path: "somerepos",
ref: "master",
submodules: defaultSubmodules,
timeout: defaultTimeout,
},
"t12": {
input: "somerepos?version=master&timeout=0s",
path: "somerepos",
ref: "master",
submodules: defaultSubmodules,
timeout: defaultTimeout,
},
"t13": {
input: "somerepos?version=master&timeout=61",
path: "somerepos",
ref: "master",
submodules: defaultSubmodules,
timeout: 61 * time.Second,
},
"t14": {
input: "somerepos?version=master&timeout=1m1s",
path: "somerepos",
ref: "master",
submodules: defaultSubmodules,
timeout: 61 * time.Second,
},
"t15": {
input: "somerepos?version=master&submodules=false&timeout=1m1s",
path: "somerepos",
ref: "master",
submodules: false,
timeout: 61 * time.Second,
},
}
for _, testcase := range testcases {
path, ref := peelQuery(testcase.input)
if path != testcase.expect[0] || ref != testcase.expect[1] {
t.Errorf("peelQuery: expected (%s, %s) got (%s, %s) on %s", testcase.expect[0], testcase.expect[1], path, ref, testcase.input)
}
for tn, tc := range testcases {
t.Run(tn, func(t *testing.T) {
path, ref, timeout, submodules := peelQuery(tc.input)
assert.Equal(t, tc.path, path, "path mismatch")
assert.Equal(t, tc.ref, ref, "ref mismatch")
assert.Equal(t, tc.timeout, timeout, "timeout mismatch")
assert.Equal(t, tc.submodules, submodules, "submodules mismatch")
})
}
}

View File

@@ -9,8 +9,8 @@ import (
"sigs.k8s.io/kustomize/api/filesys"
"sigs.k8s.io/kustomize/api/loader"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
func TestLoadDefaultConfigsFromFiles(t *testing.T) {

View File

@@ -6,8 +6,8 @@ package builtinconfig
import (
"strings"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
// NameBackReferences is an association between a gvk.GVK (a ReferralTarget)

View File

@@ -7,8 +7,8 @@ import (
"reflect"
"testing"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
func TestMergeAll(t *testing.T) {

View File

@@ -8,8 +8,8 @@ import (
"testing"
. "sigs.k8s.io/kustomize/api/internal/plugins/builtinconfig"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
func TestMakeDefaultConfig(t *testing.T) {

View File

@@ -11,25 +11,26 @@ func _() {
_ = x[Unknown-0]
_ = x[AnnotationsTransformer-1]
_ = x[ConfigMapGenerator-2]
_ = x[HashTransformer-3]
_ = x[ImageTagTransformer-4]
_ = x[LabelTransformer-5]
_ = x[LegacyOrderTransformer-6]
_ = x[NamespaceTransformer-7]
_ = x[PatchJson6902Transformer-8]
_ = x[PatchStrategicMergeTransformer-9]
_ = x[PatchTransformer-10]
_ = x[PrefixSuffixTransformer-11]
_ = x[ReplicaCountTransformer-12]
_ = x[SecretGenerator-13]
_ = x[ValueAddTransformer-14]
_ = x[HelmChartInflationGenerator-15]
_ = x[ReplacementTransformer-16]
_ = x[IAMPolicyGenerator-3]
_ = x[HashTransformer-4]
_ = x[ImageTagTransformer-5]
_ = x[LabelTransformer-6]
_ = x[LegacyOrderTransformer-7]
_ = x[NamespaceTransformer-8]
_ = x[PatchJson6902Transformer-9]
_ = x[PatchStrategicMergeTransformer-10]
_ = x[PatchTransformer-11]
_ = x[PrefixSuffixTransformer-12]
_ = x[ReplicaCountTransformer-13]
_ = x[SecretGenerator-14]
_ = x[ValueAddTransformer-15]
_ = x[HelmChartInflationGenerator-16]
_ = x[ReplacementTransformer-17]
}
const _BuiltinPluginType_name = "UnknownAnnotationsTransformerConfigMapGeneratorHashTransformerImageTagTransformerLabelTransformerLegacyOrderTransformerNamespaceTransformerPatchJson6902TransformerPatchStrategicMergeTransformerPatchTransformerPrefixSuffixTransformerReplicaCountTransformerSecretGeneratorValueAddTransformerHelmChartInflationGeneratorReplacementTransformer"
const _BuiltinPluginType_name = "UnknownAnnotationsTransformerConfigMapGeneratorIAMPolicyGeneratorHashTransformerImageTagTransformerLabelTransformerLegacyOrderTransformerNamespaceTransformerPatchJson6902TransformerPatchStrategicMergeTransformerPatchTransformerPrefixSuffixTransformerReplicaCountTransformerSecretGeneratorValueAddTransformerHelmChartInflationGeneratorReplacementTransformer"
var _BuiltinPluginType_index = [...]uint16{0, 7, 29, 47, 62, 81, 97, 119, 139, 163, 193, 209, 232, 255, 270, 289, 316, 338}
var _BuiltinPluginType_index = [...]uint16{0, 7, 29, 47, 65, 80, 99, 115, 137, 157, 181, 211, 227, 250, 273, 288, 307, 334, 356}
func (i BuiltinPluginType) String() string {
if i < 0 || i >= BuiltinPluginType(len(_BuiltinPluginType_index)-1) {

View File

@@ -15,6 +15,7 @@ const (
Unknown BuiltinPluginType = iota
AnnotationsTransformer
ConfigMapGenerator
IAMPolicyGenerator
HashTransformer
ImageTagTransformer
LabelTransformer
@@ -58,6 +59,7 @@ func GetBuiltinPluginType(n string) BuiltinPluginType {
var GeneratorFactories = map[BuiltinPluginType]func() resmap.GeneratorPlugin{
ConfigMapGenerator: builtins.NewConfigMapGeneratorPlugin,
IAMPolicyGenerator: builtins.NewIAMPolicyGeneratorPlugin,
SecretGenerator: builtins.NewSecretGeneratorPlugin,
HelmChartInflationGenerator: builtins.NewHelmChartInflationGeneratorPlugin,
}

View File

@@ -94,9 +94,6 @@ TO GENERATE CODE
cd $repo/plugin/builtin
go generate ./...
See scripts/kyaml-pre-commit.sh for canonical way
to execute the above.
This creates
$repo/api/plugins/builtins/SecretGenerator.go

View File

@@ -89,7 +89,10 @@ type argsConfig struct {
func (p *ExecPlugin) processOptionalArgsFields() error {
var c argsConfig
yaml.Unmarshal(p.cfg, &c)
err := yaml.Unmarshal(p.cfg, &c)
if err != nil {
return err
}
if c.ArgsOneLiner != "" {
p.args, _ = shlex.Split(c.ArgsOneLiner)
}

View File

@@ -9,6 +9,7 @@ import (
"strings"
"testing"
"github.com/stretchr/testify/require"
"sigs.k8s.io/kustomize/api/filesys"
. "sigs.k8s.io/kustomize/api/internal/plugins/execplugin"
pLdr "sigs.k8s.io/kustomize/api/internal/plugins/loader"
@@ -21,11 +22,12 @@ import (
func TestExecPluginConfig(t *testing.T) {
fSys := filesys.MakeFsInMemory()
fSys.WriteFile("sed-input.txt", []byte(`
err := fSys.WriteFile("sed-input.txt", []byte(`
s/$FOO/foo/g
s/$BAR/bar baz/g
\ \ \
`))
require.NoError(t, err)
ldr, err := fLdr.NewLoader(
fLdr.RestrictionRootOnly, filesys.Separator, fSys)
if err != nil {
@@ -62,9 +64,10 @@ s/$BAR/bar baz/g
if err != nil {
t.Fatalf("unexpected err: %v", err)
}
p.Config(
err = p.Config(
resmap.NewPluginHelpers(ldr, pvd.GetFieldValidator(), rf, pc),
yaml)
require.NoError(t, err)
expected := "someteam.example.com/v1/sedtransformer/SedTransformer"
if !strings.HasSuffix(p.Path(), expected) {

View File

@@ -20,10 +20,10 @@ import (
"sigs.k8s.io/kustomize/api/internal/plugins/fnplugin"
"sigs.k8s.io/kustomize/api/internal/plugins/utils"
"sigs.k8s.io/kustomize/api/konfig"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/resource"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
// Loader loads plugins using a file loader (a different loader).

View File

@@ -137,7 +137,9 @@ func GetResMapWithIDAnnotation(rm resmap.ResMap) (resmap.ResMap, error) {
}
annotations := r.GetAnnotations()
annotations[idAnnotation] = string(idString)
r.SetAnnotations(annotations)
if err = r.SetAnnotations(annotations); err != nil {
return nil, err
}
}
return inputRM, nil
}
@@ -158,7 +160,10 @@ func UpdateResMapValues(pluginName string, h *resmap.PluginHelpers, output []byt
}
for _, r := range resources {
removeIDAnnotation(r) // stale--not manipulated by plugin transformers
// stale--not manipulated by plugin transformers
if err = removeIDAnnotation(r); err != nil {
return err
}
// Add to the new map, checking for duplicates
if err := newMap.Append(r); err != nil {
@@ -175,7 +180,7 @@ func UpdateResMapValues(pluginName string, h *resmap.PluginHelpers, output []byt
return err
}
if oldIdx != -1 {
rm.GetByIndex(oldIdx).ResetPrimaryData(r)
rm.GetByIndex(oldIdx).ResetRNode(r)
} else {
if err := rm.Append(r); err != nil {
return err
@@ -187,21 +192,20 @@ func UpdateResMapValues(pluginName string, h *resmap.PluginHelpers, output []byt
for _, id := range rm.AllIds() {
newIdx, _ := newMap.GetIndexOfCurrentId(id)
if newIdx == -1 {
rm.Remove(id)
if err = rm.Remove(id); err != nil {
return err
}
}
}
return nil
}
func removeIDAnnotation(r *resource.Resource) {
func removeIDAnnotation(r *resource.Resource) error {
// remove the annotation set by Kustomize to track the resource
annotations := r.GetAnnotations()
delete(annotations, idAnnotation)
if len(annotations) == 0 {
annotations = nil
}
r.SetAnnotations(annotations)
return r.SetAnnotations(annotations)
}
// UpdateResourceOptions updates the generator options for each resource in the
@@ -224,10 +228,9 @@ func UpdateResourceOptions(rm resmap.ResMap) (resmap.ResMap, error) {
}
delete(annotations, HashAnnotation)
delete(annotations, BehaviorAnnotation)
if len(annotations) == 0 {
annotations = nil
if err := r.SetAnnotations(annotations); err != nil {
return nil, err
}
r.SetAnnotations(annotations)
r.SetOptions(types.NewGenArgs(
&types.GeneratorArgs{
Behavior: behavior,

View File

@@ -10,6 +10,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sigs.k8s.io/kustomize/api/filesys"
"sigs.k8s.io/kustomize/api/konfig"
"sigs.k8s.io/kustomize/api/provider"
@@ -45,7 +46,9 @@ func makeConfigMap(rf *resource.Factory, name, behavior string, hashValue *strin
annotations[HashAnnotation] = *hashValue
}
if len(annotations) > 0 {
r.SetAnnotations(annotations)
if err := r.SetAnnotations(annotations); err != nil {
panic(err)
}
}
return r
}
@@ -84,8 +87,10 @@ func TestUpdateResourceOptions(t *testing.T) {
}
for i, c := range cases {
name := fmt.Sprintf("test%d", i)
in.Append(makeConfigMap(rf, name, c.behavior, c.hashValue))
expected.Append(makeConfigMapOptions(rf, name, c.behavior, !c.needsHash))
err := in.Append(makeConfigMap(rf, name, c.behavior, c.hashValue))
require.NoError(t, err)
err = expected.Append(makeConfigMapOptions(rf, name, c.behavior, !c.needsHash))
require.NoError(t, err)
}
actual, err := UpdateResourceOptions(in)
assert.NoError(t, err)
@@ -103,10 +108,9 @@ func TestUpdateResourceOptionsWithInvalidHashAnnotationValues(t *testing.T) {
for i, c := range cases {
name := fmt.Sprintf("test%d", i)
in := resmap.New()
in.Append(makeConfigMap(rf, name, "", &c))
_, err := UpdateResourceOptions(in)
if err == nil {
t.Errorf("expected error from value %q", c)
}
err := in.Append(makeConfigMap(rf, name, "", &c))
require.NoError(t, err)
_, err = UpdateResourceOptions(in)
require.Error(t, err)
}
}

View File

@@ -303,16 +303,18 @@ func (kt *KustTarget) runValidators(ra *accumulator.ResAccumulator) error {
if err != nil {
return err
}
new := ra.ResMap().DeepCopy()
kt.removeValidatedByLabel(new)
if err = orignal.ErrorIfNotEqualSets(new); err != nil {
newMap := ra.ResMap().DeepCopy()
if err = kt.removeValidatedByLabel(newMap); err != nil {
return err
}
if err = orignal.ErrorIfNotEqualSets(newMap); err != nil {
return fmt.Errorf("validator shouldn't modify the resource map: %v", err)
}
}
return nil
}
func (kt *KustTarget) removeValidatedByLabel(rm resmap.ResMap) {
func (kt *KustTarget) removeValidatedByLabel(rm resmap.ResMap) error {
resources := rm.Resources()
for _, r := range resources {
labels := r.GetLabels()
@@ -320,12 +322,11 @@ func (kt *KustTarget) removeValidatedByLabel(rm resmap.ResMap) {
continue
}
delete(labels, konfig.ValidatedByLabelKey)
if len(labels) == 0 {
r.SetLabels(nil)
} else {
r.SetLabels(labels)
if err := r.SetLabels(labels); err != nil {
return err
}
}
return nil
}
// accumulateResources fills the given resourceAccumulator

View File

@@ -8,9 +8,9 @@ import (
"strings"
"testing"
"sigs.k8s.io/kustomize/api/resid"
kusttest_test "sigs.k8s.io/kustomize/api/testutils/kusttest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
// To simplify tests, these vars specified in alphabetical order.

View File

@@ -5,18 +5,50 @@ package utils
import "strings"
// PathSplitter splits a slash delimited string, permitting escaped slashes.
func PathSplitter(path string) []string {
ps := strings.Split(path, "/")
// TODO: Move these to kyaml
// PathSplitter splits a delimited string, permitting escaped delimiters.
func PathSplitter(path string, delimiter string) []string {
ps := strings.Split(path, delimiter)
var res []string
res = append(res, ps[0])
for i := 1; i < len(ps); i++ {
last := len(res) - 1
if strings.HasSuffix(res[last], `\`) {
res[last] = strings.TrimSuffix(res[last], `\`) + "/" + ps[i]
res[last] = strings.TrimSuffix(res[last], `\`) + delimiter + ps[i]
} else {
res = append(res, ps[i])
}
}
return res
}
// SmarterPathSplitter splits a path, retaining bracketed list entry identifiers.
// E.g. [name=com.foo.someapp] survives as one thing after splitting
// "spec.template.spec.containers.[name=com.foo.someapp].image"
// See kyaml/yaml/match.go for use of list entry identifiers.
// This function uses `PathSplitter`, so it respects list entry identifiers
// and escaped delimiters.
func SmarterPathSplitter(path string, delimiter string) []string {
var result []string
split := PathSplitter(path, delimiter)
for i := 0; i < len(split); i++ {
elem := split[i]
if strings.HasPrefix(elem, "[") && !strings.HasSuffix(elem, "]") {
// continue until we find the matching "]"
bracketed := []string{elem}
for i < len(split)-1 {
i++
bracketed = append(bracketed, split[i])
if strings.HasSuffix(split[i], "]") {
break
}
}
result = append(result, strings.Join(bracketed, delimiter))
} else {
result = append(result, elem)
}
}
return result
}

View File

@@ -44,6 +44,47 @@ func TestPathSplitter(t *testing.T) {
"nginx.ingress.kubernetes.io/auth-secret"},
},
} {
assert.Equal(t, tc.exp, PathSplitter(tc.path))
assert.Equal(t, tc.exp, PathSplitter(tc.path, "/"))
}
}
func TestSmarterPathSplitter(t *testing.T) {
testCases := map[string]struct {
input string
expected []string
}{
"simple": {
input: "spec.replicas",
expected: []string{"spec", "replicas"},
},
"sequence": {
input: "spec.data.[name=first].key",
expected: []string{"spec", "data", "[name=first]", "key"},
},
"key, value with . prefix": {
input: "spec.data.[.name=.first].key",
expected: []string{"spec", "data", "[.name=.first]", "key"},
},
"key, value with . suffix": {
input: "spec.data.[name.=first.].key",
expected: []string{"spec", "data", "[name.=first.]", "key"},
},
"multiple '.' in value": {
input: "spec.data.[name=f.i.r.s.t.].key",
expected: []string{"spec", "data", "[name=f.i.r.s.t.]", "key"},
},
"with escaped delimiter": {
input: `spec\.replicas`,
expected: []string{`spec.replicas`},
},
"unmatched bracket": {
input: "spec.data.[name=f.i.[r.s.t..key",
expected: []string{"spec", "data", "[name=f.i.[r.s.t..key"},
},
}
for tn, tc := range testCases {
t.Run(tn, func(t *testing.T) {
assert.Equal(t, tc.expected, SmarterPathSplitter(tc.input, "."))
})
}
}

View File

@@ -3,9 +3,6 @@
package builtinpluginconsts
// TODO: rename 'fieldSpecs' to 'referrers' for clarity.
// This will, however, break anyone using a custom config.
const (
nameReferenceFieldSpecs = `
nameReference:

View File

@@ -2,5 +2,6 @@
// SPDX-License-Identifier: Apache-2.0
// Package konfig provides configuration methods and constants
// for the kustomize API.
// for the kustomize API, e.g. the set of file names to look for
// to identify a kustomization root.
package konfig

View File

@@ -9,7 +9,7 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sigs.k8s.io/kustomize/api/filesys"
"sigs.k8s.io/kustomize/api/types"
)
@@ -18,11 +18,11 @@ func TestDefaultAbsPluginHome_NoKustomizePluginHomeEnv(t *testing.T) {
fSys := filesys.MakeFsInMemory()
keep, isSet := os.LookupEnv(KustomizePluginHomeEnv)
if isSet {
_ = os.Unsetenv(KustomizePluginHomeEnv)
unsetenv(t, KustomizePluginHomeEnv)
}
_, err := DefaultAbsPluginHome(fSys)
if isSet {
os.Setenv(KustomizePluginHomeEnv, keep)
setenv(t, KustomizePluginHomeEnv, keep)
}
if err == nil {
t.Fatalf("expected err")
@@ -43,13 +43,13 @@ func TestDefaultAbsPluginHome_NoKustomizePluginHomeEnv(t *testing.T) {
func TestDefaultAbsPluginHome_EmptyKustomizePluginHomeEnv(t *testing.T) {
keep, isSet := os.LookupEnv(KustomizePluginHomeEnv)
os.Setenv(KustomizePluginHomeEnv, "")
setenv(t, KustomizePluginHomeEnv, "")
_, err := DefaultAbsPluginHome(filesys.MakeFsInMemory())
if !isSet {
_ = os.Unsetenv(KustomizePluginHomeEnv)
unsetenv(t, KustomizePluginHomeEnv)
} else {
_ = os.Setenv(KustomizePluginHomeEnv, keep)
setenv(t, KustomizePluginHomeEnv, keep)
}
if err == nil {
t.Fatalf("expected err")
@@ -65,16 +65,15 @@ func TestDefaultAbsPluginHome_WithKustomizePluginHomeEnv(t *testing.T) {
keep, isSet := os.LookupEnv(KustomizePluginHomeEnv)
if !isSet {
keep = "whatever"
os.Setenv(KustomizePluginHomeEnv, keep)
setenv(t, KustomizePluginHomeEnv, keep)
}
fSys.Mkdir(keep)
err := fSys.Mkdir(keep)
require.NoError(t, err)
h, err := DefaultAbsPluginHome(fSys)
if !isSet {
_ = os.Unsetenv(KustomizePluginHomeEnv)
}
if err != nil {
t.Fatalf("unexpected err: %v", err)
unsetenv(t, KustomizePluginHomeEnv)
}
require.NoError(t, err)
if h != keep {
t.Fatalf("unexpected config dir: %s", h)
}
@@ -85,13 +84,14 @@ func TestDefaultAbsPluginHomeWithXdg(t *testing.T) {
keep, isSet := os.LookupEnv(XdgConfigHomeEnv)
if !isSet {
keep = "whatever"
os.Setenv(XdgConfigHomeEnv, keep)
setenv(t, XdgConfigHomeEnv, keep)
}
configDir := filepath.Join(keep, ProgramName, RelPluginHome)
fSys.Mkdir(configDir)
err := fSys.Mkdir(configDir)
require.NoError(t, err)
h, err := DefaultAbsPluginHome(fSys)
if !isSet {
_ = os.Unsetenv(XdgConfigHomeEnv)
unsetenv(t, XdgConfigHomeEnv)
}
if err != nil {
t.Fatalf("unexpected err: %v", err)
@@ -105,11 +105,11 @@ func TestDefaultAbsPluginHomeNoConfig(t *testing.T) {
fSys := filesys.MakeFsInMemory()
keep, isSet := os.LookupEnv(XdgConfigHomeEnv)
if isSet {
_ = os.Unsetenv(XdgConfigHomeEnv)
unsetenv(t, XdgConfigHomeEnv)
}
_, err := DefaultAbsPluginHome(fSys)
if isSet {
os.Setenv(XdgConfigHomeEnv, keep)
setenv(t, XdgConfigHomeEnv, keep)
}
if err == nil {
t.Fatalf("expected err")
@@ -121,13 +121,13 @@ func TestDefaultAbsPluginHomeNoConfig(t *testing.T) {
func TestDefaultAbsPluginHomeEmptyXdgConfig(t *testing.T) {
keep, isSet := os.LookupEnv(XdgConfigHomeEnv)
os.Setenv(XdgConfigHomeEnv, "")
setenv(t, XdgConfigHomeEnv, "")
if isSet {
_ = os.Unsetenv(XdgConfigHomeEnv)
unsetenv(t, XdgConfigHomeEnv)
}
_, err := DefaultAbsPluginHome(filesys.MakeFsInMemory())
if isSet {
os.Setenv(XdgConfigHomeEnv, keep)
setenv(t, XdgConfigHomeEnv, keep)
}
if err == nil {
t.Fatalf("expected err")
@@ -142,14 +142,16 @@ func TestDefaultAbsPluginHomeNoXdgWithDotConfig(t *testing.T) {
fSys := filesys.MakeFsInMemory()
configDir := filepath.Join(
HomeDir(), XdgConfigHomeEnvDefault, ProgramName, RelPluginHome)
fSys.Mkdir(configDir)
err := fSys.Mkdir(configDir)
require.NoError(t, err)
keep, isSet := os.LookupEnv(XdgConfigHomeEnv)
if isSet {
_ = os.Unsetenv(XdgConfigHomeEnv)
unsetenv(t, XdgConfigHomeEnv)
}
s, _ := DefaultAbsPluginHome(fSys)
s, err := DefaultAbsPluginHome(fSys)
require.NoError(t, err)
if isSet {
os.Setenv(XdgConfigHomeEnv, keep)
setenv(t, XdgConfigHomeEnv, keep)
}
if s != configDir {
t.Fatalf("unexpected config dir: %s", s)
@@ -160,16 +162,26 @@ func TestDefaultAbsPluginHomeNoXdgJustHomeDir(t *testing.T) {
fSys := filesys.MakeFsInMemory()
configDir := filepath.Join(
HomeDir(), ProgramName, RelPluginHome)
fSys.Mkdir(configDir)
err := fSys.Mkdir(configDir)
require.NoError(t, err)
keep, isSet := os.LookupEnv(XdgConfigHomeEnv)
if isSet {
_ = os.Unsetenv(XdgConfigHomeEnv)
unsetenv(t, XdgConfigHomeEnv)
}
s, _ := DefaultAbsPluginHome(fSys)
s, err := DefaultAbsPluginHome(fSys)
require.NoError(t, err)
if isSet {
os.Setenv(XdgConfigHomeEnv, keep)
setenv(t, XdgConfigHomeEnv, keep)
}
if s != configDir {
t.Fatalf("unexpected config dir: %s", s)
}
}
func setenv(t *testing.T, key, value string) {
require.NoError(t, os.Setenv(key, value))
}
func unsetenv(t *testing.T, key string) {
require.NoError(t, os.Unsetenv(key))
}

View File

@@ -90,6 +90,7 @@ func skipIfNoDocker(t *testing.T) {
}
func TestFnContainerGenerator(t *testing.T) {
t.Skip("wait for #3881")
skipIfNoDocker(t)
// Function plugins should not need the env setup done by MakeEnhancedHarness
@@ -306,6 +307,7 @@ spec:
}
func TestFnContainerTransformer(t *testing.T) {
t.Skip("wait for #3881")
skipIfNoDocker(t)
// Function plugins should not need the env setup done by MakeEnhancedHarness

View File

@@ -81,3 +81,156 @@ helmCharts:
m := th.Run(th.GetRoot(), th.MakeOptionsPluginsEnabled())
th.AssertActualEqualsExpected(m, expectedHelm)
}
// Last mile helm - show how kustomize puts helm charts into different
// namespaces with different customizations.
func TestHelmChartProdVsDev(t *testing.T) {
th := kusttest_test.MakeEnhancedHarnessWithTmpRoot(t)
defer th.Reset()
if err := th.ErrIfNoHelm(); err != nil {
t.Skip("skipping: " + err.Error())
}
dirBase := th.MkDir("base")
dirProd := th.MkDir("prod")
dirDev := th.MkDir("dev")
dirBoth := th.MkDir("both")
th.WriteK(dirBase, `
helmCharts:
- name: minecraft
repo: https://itzg.github.io/minecraft-server-charts
version: 3.1.3
releaseName: test
`)
th.WriteK(dirProd, `
namespace: prod
namePrefix: myProd-
resources:
- ../base
`)
th.WriteK(dirDev, `
namespace: dev
namePrefix: myDev-
resources:
- ../base
`)
th.WriteK(dirBoth, `
resources:
- ../dev
- ../prod
`)
// Base unchanged
m := th.Run(dirBase, th.MakeOptionsPluginsEnabled())
th.AssertActualEqualsExpected(m, expectedHelm)
// Prod has a "prod" namespace and a prefix.
m = th.Run(dirProd, th.MakeOptionsPluginsEnabled())
th.AssertActualEqualsExpected(m, `
apiVersion: v1
data:
rcon-password: Q0hBTkdFTUUh
kind: Secret
metadata:
labels:
app: test-minecraft
chart: minecraft-3.1.3
heritage: Helm
release: test
name: myProd-test-minecraft
namespace: prod
type: Opaque
---
apiVersion: v1
kind: Service
metadata:
labels:
app: test-minecraft
chart: minecraft-3.1.3
heritage: Helm
release: test
name: myProd-test-minecraft
namespace: prod
spec:
ports:
- name: minecraft
port: 25565
protocol: TCP
targetPort: minecraft
selector:
app: test-minecraft
type: ClusterIP
`)
// Both has two namespaces.
m = th.Run(dirBoth, th.MakeOptionsPluginsEnabled())
th.AssertActualEqualsExpected(m, `
apiVersion: v1
data:
rcon-password: Q0hBTkdFTUUh
kind: Secret
metadata:
labels:
app: test-minecraft
chart: minecraft-3.1.3
heritage: Helm
release: test
name: myDev-test-minecraft
namespace: dev
type: Opaque
---
apiVersion: v1
kind: Service
metadata:
labels:
app: test-minecraft
chart: minecraft-3.1.3
heritage: Helm
release: test
name: myDev-test-minecraft
namespace: dev
spec:
ports:
- name: minecraft
port: 25565
protocol: TCP
targetPort: minecraft
selector:
app: test-minecraft
type: ClusterIP
---
apiVersion: v1
data:
rcon-password: Q0hBTkdFTUUh
kind: Secret
metadata:
labels:
app: test-minecraft
chart: minecraft-3.1.3
heritage: Helm
release: test
name: myProd-test-minecraft
namespace: prod
type: Opaque
---
apiVersion: v1
kind: Service
metadata:
labels:
app: test-minecraft
chart: minecraft-3.1.3
heritage: Helm
release: test
name: myProd-test-minecraft
namespace: prod
spec:
ports:
- name: minecraft
port: 25565
protocol: TCP
targetPort: minecraft
selector:
app: test-minecraft
type: ClusterIP
`)
}

View File

@@ -0,0 +1,124 @@
package krusty_test
import (
"testing"
kusttest_test "sigs.k8s.io/kustomize/api/testutils/kusttest"
)
func TestGkeGenerator(t *testing.T) {
th := kusttest_test.MakeEnhancedHarness(t)
defer th.Reset()
th.WriteK(".", `
generators:
- |-
apiVersion: builtin
kind: IAMPolicyGenerator
metadata:
name: my-gke-generator
cloud: gke
kubernetesService:
name: k8s-sa-name
serviceAccount:
name: gsa-name
projectId: project-id
`)
expected := `
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
iam.gke.io/gcp-service-account: gsa-name@project-id.iam.gserviceaccount.com
name: k8s-sa-name
`
m := th.Run(".", th.MakeDefaultOptions())
th.AssertActualEqualsExpected(m, expected)
}
func TestGkeGeneratorWithNamespace(t *testing.T) {
th := kusttest_test.MakeEnhancedHarness(t)
defer th.Reset()
th.WriteK(".", `
generators:
- |-
apiVersion: builtin
kind: IAMPolicyGenerator
metadata:
name: my-gke-generator
cloud: gke
kubernetesService:
namespace: k8s-namespace
name: k8s-sa-name
serviceAccount:
name: gsa-name
projectId: project-id
`)
expected := `
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
iam.gke.io/gcp-service-account: gsa-name@project-id.iam.gserviceaccount.com
name: k8s-sa-name
namespace: k8s-namespace
`
m := th.Run(".", th.MakeDefaultOptions())
th.AssertActualEqualsExpected(m, expected)
}
func TestGkeGeneratorWithTwo(t *testing.T) {
th := kusttest_test.MakeEnhancedHarness(t)
defer th.Reset()
th.WriteK(".", `
generators:
- gkegenerator1.yaml
- gkegenerator2.yaml
`)
th.WriteF("gkegenerator1.yaml", `
apiVersion: builtin
kind: IAMPolicyGenerator
metadata:
name: my-gke-generator1
cloud: gke
kubernetesService:
namespace: k8s-namespace-1
name: k8s-sa-name-1
serviceAccount:
name: gsa-name-1
projectId: project-id-1
`)
th.WriteF("gkegenerator2.yaml", `
apiVersion: builtin
kind: IAMPolicyGenerator
metadata:
name: my-gke-generator2
cloud: gke
kubernetesService:
name: k8s-sa-name-2
serviceAccount:
name: gsa-name-2
projectId: project-id-2
`)
expected := `
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
iam.gke.io/gcp-service-account: gsa-name-1@project-id-1.iam.gserviceaccount.com
name: k8s-sa-name-1
namespace: k8s-namespace-1
---
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
iam.gke.io/gcp-service-account: gsa-name-2@project-id-2.iam.gserviceaccount.com
name: k8s-sa-name-2
`
m := th.Run(".", th.MakeDefaultOptions())
th.AssertActualEqualsExpected(m, expected)
}

View File

@@ -90,19 +90,25 @@ func (b *Kustomizer) Run(
return nil, err
}
if b.options.DoLegacyResourceSort {
builtins.NewLegacyOrderTransformerPlugin().Transform(m)
err = builtins.NewLegacyOrderTransformerPlugin().Transform(m)
if err != nil {
return nil, err
}
}
if b.options.AddManagedbyLabel {
t := builtins.LabelTransformerPlugin{
Labels: map[string]string{
konfig.ManagedbyLabelKey: fmt.Sprintf(
"kustomize-%s", provenance.GetProvenance().Semver())},
konfig.ManagedbyLabelKey: fmt.Sprintf("kustomize-%s", provenance.GetProvenance().Semver()),
},
FieldSpecs: []types.FieldSpec{{
Path: "metadata/labels",
CreateIfNotPresent: true,
}},
}
t.Transform(m)
err = t.Transform(m)
if err != nil {
return nil, err
}
}
m.RemoveBuildAnnotations()
return m, nil

View File

@@ -6,9 +6,319 @@ package krusty_test
import (
"testing"
"github.com/stretchr/testify/assert"
kusttest_test "sigs.k8s.io/kustomize/api/testutils/kusttest"
)
func TestPatchesInOneFile(t *testing.T) {
th := kusttest_test.MakeHarness(t)
th.WriteK("base", `
resources:
- namespace.yaml
- deployment-controller-manager.yaml
- deployment-audit-manager.yaml
`)
th.WriteF("base/namespace.yaml", `
apiVersion: v1
kind: Namespace
metadata:
labels:
control-plane: controller-manager
admission.gatekeeper.sh/ignore: no-self-managing
name: system
`)
th.WriteF("base/deployment-controller-manager.yaml", `
apiVersion: apps/v1
kind: Deployment
metadata:
name: controller-manager
namespace: system
labels:
control-plane: controller-manager
gatekeeper.sh/operation: webhook
spec:
selector:
matchLabels:
control-plane: controller-manager
gatekeeper.sh/operation: webhook
replicas: 3
template:
metadata:
annotations:
container.seccomp.security.alpha.kubernetes.io/manager: runtime/default
labels:
control-plane: controller-manager
gatekeeper.sh/operation: webhook
spec:
containers:
- command:
- /manager
args:
- "--port=8443"
- "--logtostderr"
- "--exempt-namespace=gatekeeper-system"
- "--operation=webhook"
image: openpolicyagent/gatekeeper:v3.4.0
imagePullPolicy: Always
name: manager
terminationGracePeriodSeconds: 60
nodeSelector:
kubernetes.io/os: linux
priorityClassName: system-cluster-critical
`)
th.WriteF("base/deployment-audit-manager.yaml", `
apiVersion: apps/v1
kind: Deployment
metadata:
name: audit
namespace: system
labels:
control-plane: audit-controller
gatekeeper.sh/operation: audit
spec:
selector:
matchLabels:
control-plane: audit-controller
gatekeeper.sh/operation: audit
replicas: 1
template:
metadata:
labels:
control-plane: audit-controller
gatekeeper.sh/operation: audit
annotations:
container.seccomp.security.alpha.kubernetes.io/manager: runtime/default
spec:
automountServiceAccountToken: true
containers:
- args:
- --operation=audit
- --operation=status
- --logtostderr
command:
- /manager
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
image: openpolicyagent/gatekeeper:v3.4.0
imagePullPolicy: Always
livenessProbe:
httpGet:
path: /healthz
port: 9090
name: manager
serviceAccountName: gatekeeper-admin
terminationGracePeriodSeconds: 60
nodeSelector:
kubernetes.io/os: linux
priorityClassName: system-cluster-critical
`)
const imagePatchAuditManager = `
apiVersion: apps/v1
kind: Deployment
metadata:
name: audit
namespace: system
spec:
template:
spec:
containers:
- image: AUDIT_IMAGE
name: manager
args:
- --port=8443
- --logtostderr
- --emit-admission-events
- --exempt-namespace=gatekeeper-system
- --operation=webhook
- --disable-opa-builtin=http.send
`
const imagePatchControllerManager = `
apiVersion: apps/v1
kind: Deployment
metadata:
name: controller-manager
namespace: system
spec:
template:
spec:
containers:
- image: CONTROLLER_IMAGE
name: manager
args:
- --emit-audit-events
- --operation=audit
- --operation=status
- --logtostderr
`
th.WriteF(
"overlay/image_patch_audit_manager.yaml",
imagePatchAuditManager)
th.WriteF(
"overlay/image_patch_controller_manager.yaml",
imagePatchControllerManager)
const expected = `
apiVersion: v1
kind: Namespace
metadata:
labels:
admission.gatekeeper.sh/ignore: no-self-managing
control-plane: controller-manager
name: system
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
control-plane: controller-manager
gatekeeper.sh/operation: webhook
name: controller-manager
namespace: system
spec:
replicas: 3
selector:
matchLabels:
control-plane: controller-manager
gatekeeper.sh/operation: webhook
template:
metadata:
annotations:
container.seccomp.security.alpha.kubernetes.io/manager: runtime/default
labels:
control-plane: controller-manager
gatekeeper.sh/operation: webhook
spec:
containers:
- args:
- --emit-audit-events
- --operation=audit
- --operation=status
- --logtostderr
command:
- /manager
image: CONTROLLER_IMAGE
imagePullPolicy: Always
name: manager
nodeSelector:
kubernetes.io/os: linux
priorityClassName: system-cluster-critical
terminationGracePeriodSeconds: 60
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
control-plane: audit-controller
gatekeeper.sh/operation: audit
name: audit
namespace: system
spec:
replicas: 1
selector:
matchLabels:
control-plane: audit-controller
gatekeeper.sh/operation: audit
template:
metadata:
annotations:
container.seccomp.security.alpha.kubernetes.io/manager: runtime/default
labels:
control-plane: audit-controller
gatekeeper.sh/operation: audit
spec:
automountServiceAccountToken: true
containers:
- args:
- --port=8443
- --logtostderr
- --emit-admission-events
- --exempt-namespace=gatekeeper-system
- --operation=webhook
- --disable-opa-builtin=http.send
command:
- /manager
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
image: AUDIT_IMAGE
imagePullPolicy: Always
livenessProbe:
httpGet:
path: /healthz
port: 9090
name: manager
nodeSelector:
kubernetes.io/os: linux
priorityClassName: system-cluster-critical
serviceAccountName: gatekeeper-admin
terminationGracePeriodSeconds: 60
`
// Technique 1: "patchesStrategicMerge:" field, two patch files.
th.WriteK("overlay", `
resources:
- ../base
patchesStrategicMerge:
- image_patch_controller_manager.yaml
- image_patch_audit_manager.yaml
`)
m := th.Run("overlay", th.MakeDefaultOptions())
th.AssertActualEqualsExpected(m, expected)
// Technique 2: "patches:" field, two patch files.
th.WriteK("overlay", `
resources:
- ../base
patches:
- path: image_patch_controller_manager.yaml
- path: image_patch_audit_manager.yaml
`)
m = th.Run("overlay", th.MakeDefaultOptions())
th.AssertActualEqualsExpected(m, expected)
// Technique 3: "patchesStrategicMerge:" field, one patch file.
th.WriteK("overlay", `
resources:
- ../base
patchesStrategicMerge:
- twoPatchesInOneFile.yaml
`)
th.WriteF(
"overlay/twoPatchesInOneFile.yaml",
imagePatchAuditManager+"\n---\n"+imagePatchControllerManager)
m = th.Run("overlay", th.MakeDefaultOptions())
th.AssertActualEqualsExpected(m, expected)
// Technique 4: "patches:" field, one patch file. Fails.
th.WriteK("overlay", `
resources:
- ../base
patches:
- path: twoPatchesInOneFile.yaml
`)
err := th.RunWithErr("overlay", th.MakeDefaultOptions())
assert.Error(t, err)
// This should fail, because the semantics of the `patches` field.
// That field allows specific patch targeting to a list of targets,
// while the `patchesStrategicMerge` field accepts patches that
// implicitly identify their targets via GVKN.
assert.Contains(t, err.Error(), "unable to parse SM or JSON patch from ")
}
func TestRemoveEmptyDirWithNullFieldInSmp(t *testing.T) {
th := kusttest_test.MakeHarness(t)
th.WriteK(".", `

View File

@@ -7,6 +7,7 @@ import (
"reflect"
"testing"
"github.com/stretchr/testify/require"
"sigs.k8s.io/kustomize/api/filesys"
ldr "sigs.k8s.io/kustomize/api/loader"
valtest_test "sigs.k8s.io/kustomize/api/testutils/valtest"
@@ -83,7 +84,8 @@ func TestKeyValuesFromFileSources(t *testing.T) {
}
fSys := filesys.MakeFsInMemory()
fSys.WriteFile("/files/app-init.ini", []byte("FOO=bar"))
err := fSys.WriteFile("/files/app-init.ini", []byte("FOO=bar"))
require.NoError(t, err)
kvl := makeKvLoader(fSys)
for _, tc := range tests {
kvs, err := kvl.keyValuesFromFileSources(tc.sources)

View File

@@ -19,7 +19,7 @@ package resmap
import (
"sort"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/kyaml/resid"
)
// IdSlice implements the sort interface.

View File

@@ -21,7 +21,7 @@ import (
"sort"
"testing"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/kyaml/resid"
)
func TestLess(t *testing.T) {

View File

@@ -7,9 +7,10 @@ package resmap
import (
"sigs.k8s.io/kustomize/api/ifc"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/resource"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
@@ -168,21 +169,20 @@ type ResMap interface {
// GroupedByCurrentNamespace returns a map of namespace
// to a slice of *Resource in that namespace.
// Resources for whom IsNamespaceableKind is false are
// are not included at all (see NonNamespaceable).
// Cluster-scoped Resources are not included (see ClusterScoped).
// Resources with an empty namespace are placed
// in the resid.DefaultNamespace entry.
GroupedByCurrentNamespace() map[string][]*resource.Resource
// GroupByOrginalNamespace performs as GroupByNamespace
// GroupedByOriginalNamespace performs as GroupByNamespace
// but use the original namespace instead of the current
// one to perform the grouping.
GroupedByOriginalNamespace() map[string][]*resource.Resource
// NonNamespaceable returns a slice of resources that
// ClusterScoped returns a slice of resources that
// cannot be placed in a namespace, e.g.
// Node, ClusterRole, Namespace itself, etc.
NonNamespaceable() []*resource.Resource
ClusterScoped() []*resource.Resource
// AllIds returns all CurrentIds.
AllIds() []resid.ResId
@@ -254,4 +254,14 @@ type ResMap interface {
// RemoveBuildAnnotations removes annotations created by the build process.
RemoveBuildAnnotations()
// ApplyFilter applies an RNode filter to all Resources in the ResMap.
// TODO: Send/recover ancillary Resource data to/from subprocesses.
// Assure that the ancillary data in Resource (everything not in the RNode)
// is sent to and re-captured from transformer subprocess (as the process
// might edit that information). One way to do this would be to solely use
// RNode metadata annotation reading and writing instead of using Resource
// struct data members, i.e. the Resource struct is replaced by RNode
// and use of (slow) k8s metadata annotations inside the RNode.
ApplyFilter(f kio.Filter) error
}

View File

@@ -6,11 +6,13 @@ package resmap
import (
"bytes"
"fmt"
"reflect"
"github.com/pkg/errors"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/resource"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/resid"
kyaml "sigs.k8s.io/kustomize/kyaml/yaml"
)
@@ -41,7 +43,7 @@ func (m *resWrangler) Clear() {
func (m *resWrangler) DropEmpties() {
var rList []*resource.Resource
for _, r := range m.rList {
if !r.IsEmpty() {
if !r.IsNilOrEmpty() {
rList = append(rList, r)
}
}
@@ -231,15 +233,15 @@ func demandOneMatch(
return nil, fmt.Errorf("no matches for %s %s", s, id)
}
// GroupedByCurrentNamespace implements ResMap.GroupByCurrentNamespace
// GroupedByCurrentNamespace implements ResMap.
func (m *resWrangler) GroupedByCurrentNamespace() map[string][]*resource.Resource {
items := m.groupedByCurrentNamespace()
delete(items, resid.TotallyNotANamespace)
return items
}
// NonNamespaceable implements ResMap.NonNamespaceable
func (m *resWrangler) NonNamespaceable() []*resource.Resource {
// ClusterScoped implements ResMap.
func (m *resWrangler) ClusterScoped() []*resource.Resource {
return m.groupedByCurrentNamespace()[resid.TotallyNotANamespace]
}
@@ -255,7 +257,7 @@ func (m *resWrangler) groupedByCurrentNamespace() map[string][]*resource.Resourc
return byNamespace
}
// GroupedByNamespace implements ResMap.GroupByOrginalNamespace
// GroupedByOriginalNamespace implements ResMap.
func (m *resWrangler) GroupedByOriginalNamespace() map[string][]*resource.Resource {
items := m.groupedByOriginalNamespace()
delete(items, resid.TotallyNotANamespace)
@@ -323,7 +325,7 @@ func (m *resWrangler) ErrorIfNotEqualSets(other ResMap) error {
"id in self matches %d in other; id: %s", len(others), id)
}
r2 := others[0]
if !r1.NodeEqual(r2) {
if !reflect.DeepEqual(r1.RNode, r2.RNode) {
return fmt.Errorf(
"nodes unequal: \n -- %s,\n -- %s\n\n--\n%#v\n------\n%#v\n",
r1, r2, r1, r2)
@@ -336,7 +338,7 @@ func (m *resWrangler) ErrorIfNotEqualSets(other ResMap) error {
return nil
}
// ErrorIfNotEqualList implements ResMap.
// ErrorIfNotEqualLists implements ResMap.
func (m *resWrangler) ErrorIfNotEqualLists(other ResMap) error {
m2, ok := other.(*resWrangler)
if !ok {
@@ -388,7 +390,7 @@ func (m *resWrangler) makeCopy(copier resCopier) ResMap {
func (m *resWrangler) SubsetThatCouldBeReferencedByResource(
referrer *resource.Resource) ResMap {
referrerId := referrer.CurId()
if !referrerId.IsNamespaceableKind() {
if referrerId.IsClusterScoped() {
// A cluster scoped resource can refer to anything.
return m
}
@@ -396,7 +398,7 @@ func (m *resWrangler) SubsetThatCouldBeReferencedByResource(
roleBindingNamespaces := getNamespacesForRoleBinding(referrer)
for _, possibleTarget := range m.rList {
id := possibleTarget.CurId()
if !id.IsNamespaceableKind() {
if id.IsClusterScoped() {
// A cluster-scoped resource can be referred to by anything.
result.append(possibleTarget)
continue
@@ -409,8 +411,7 @@ func (m *resWrangler) SubsetThatCouldBeReferencedByResource(
// The two objects are namespaced (not cluster-scoped), AND
// are in different namespaces.
// There's still a chance they can refer to each other.
ns := possibleTarget.GetNamespace()
if roleBindingNamespaces[ns] {
if roleBindingNamespaces[possibleTarget.GetNamespace()] {
result.append(possibleTarget)
}
}
@@ -424,6 +425,7 @@ func getNamespacesForRoleBinding(r *resource.Resource) map[string]bool {
if r.GetKind() != "RoleBinding" {
return result
}
//nolint staticcheck
subjects, err := r.GetSlice("subjects")
if err != nil || subjects == nil {
return result
@@ -586,7 +588,7 @@ func (m *resWrangler) Select(s types.Selector) ([]*resource.Resource, error) {
func (m *resWrangler) ToRNodeSlice() []*kyaml.RNode {
result := make([]*kyaml.RNode, len(m.rList))
for i := range m.rList {
result[i] = m.rList[i].AsRNode()
result[i] = m.rList[i].Copy()
}
return result
}
@@ -605,7 +607,7 @@ func (m *resWrangler) ApplySmPatch(
return err
}
}
if !res.IsEmpty() {
if !res.IsNilOrEmpty() {
list = append(list, res)
}
}
@@ -618,3 +620,43 @@ func (m *resWrangler) RemoveBuildAnnotations() {
r.RemoveBuildAnnotations()
}
}
// ApplyFilter implements ResMap.
func (m *resWrangler) ApplyFilter(f kio.Filter) error {
reverseLookup := make(map[*kyaml.RNode]*resource.Resource, len(m.rList))
nodes := make([]*kyaml.RNode, len(m.rList))
for i, r := range m.rList {
ptr := &(r.RNode)
nodes[i] = ptr
reverseLookup[ptr] = r
}
// The filter can modify nodes, but also delete and create them.
// The filtered list might be smaller or larger than the nodes list.
filtered, err := f.Filter(nodes)
if err != nil {
return err
}
// Rebuild the resmap from the filtered RNodes.
var nRList []*resource.Resource
for _, rn := range filtered {
if rn.IsNilOrEmpty() {
// A node might make it through the filter as an object,
// but still be empty. Drop such entries.
continue
}
res, ok := reverseLookup[rn]
if !ok {
// A node was created; make a Resource to wrap it.
res = &resource.Resource{
RNode: *rn,
// Leave remaining fields empty.
// At at time of writing, seeking to eliminate those fields.
// Alternatively, could just return error on creation attempt
// until remaining fields eliminated.
}
}
nRList = append(nRList, res)
}
m.rList = nRList
return nil
}

View File

@@ -11,12 +11,16 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"sigs.k8s.io/kustomize/api/filters/labels"
"sigs.k8s.io/kustomize/api/provider"
"sigs.k8s.io/kustomize/api/resid"
. "sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/resource"
kusttest_test "sigs.k8s.io/kustomize/api/testutils/kusttest"
resmaptest_test "sigs.k8s.io/kustomize/api/testutils/resmaptest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
var depProvider = provider.NewDefaultDepProvider()
@@ -192,7 +196,7 @@ metadata:
}
func TestGetMatchingResourcesByCurrentId(t *testing.T) {
cmap := resid.Gvk{Version: "v1", Kind: "ConfigMap"}
cmap := resid.NewGvk("", "v1", "ConfigMap")
r1 := rf.FromMap(
map[string]interface{}{
@@ -1144,6 +1148,186 @@ func addNamespace(namespace string, base string) string {
return res
}
// DeleteOddsFilter deletes the odd entries, removing nodes.
// This is a ridiculous filter for testing.
type DeleteOddsFilter struct{}
func (f DeleteOddsFilter) Filter(
nodes []*yaml.RNode) (result []*yaml.RNode, err error) {
for i := range nodes {
if i%2 == 0 {
// Keep the even entries, drop the odd entries.
result = append(result, nodes[i])
}
}
return
}
// CloneOddsFilter deletes even entries and clones odd entries,
// making new nodes.
// This is a ridiculous filter for testing.
type CloneOddsFilter struct{}
func (f CloneOddsFilter) Filter(
nodes []*yaml.RNode) (result []*yaml.RNode, err error) {
for i := range nodes {
if i%2 != 0 {
newNode := nodes[i].Copy()
// Add suffix to the name, so that it's unique (w/r to this test).
newNode.SetName(newNode.GetName() + "Clone")
// Return a ptr to the copy.
result = append(result, nodes[i], newNode)
}
}
return
}
func TestApplyFilter(t *testing.T) {
tests := map[string]struct {
input string
f kio.Filter
expected string
}{
"labels": {
input: `
apiVersion: example.com/v1
kind: Beans
metadata:
name: myBeans
---
apiVersion: example.com/v1
kind: Franks
metadata:
name: myFranks
`,
f: labels.Filter{
Labels: map[string]string{
"a": "foo",
"b": "bar",
},
FsSlice: types.FsSlice{
{
Gvk: resid.NewGvk("example.com", "v1", "Beans"),
Path: "metadata/labels",
CreateIfNotPresent: true,
},
},
},
expected: `
apiVersion: example.com/v1
kind: Beans
metadata:
labels:
a: foo
b: bar
name: myBeans
---
apiVersion: example.com/v1
kind: Franks
metadata:
name: myFranks
`,
},
"deleteOddNodes": {
input: `
apiVersion: example.com/v1
kind: Zero
metadata:
name: r0
---
apiVersion: example.com/v1
kind: One
metadata:
name: r1
---
apiVersion: example.com/v1
kind: Two
metadata:
name: r2
---
apiVersion: example.com/v1
kind: Three
metadata:
name: r3
`,
f: DeleteOddsFilter{},
expected: `
apiVersion: example.com/v1
kind: Zero
metadata:
name: r0
---
apiVersion: example.com/v1
kind: Two
metadata:
name: r2
`,
},
"cloneOddNodes": {
// input list has five entries
input: `
apiVersion: example.com/v1
kind: Zero
metadata:
name: r0
---
apiVersion: example.com/v1
kind: One
metadata:
name: r1
---
apiVersion: example.com/v1
kind: Two
metadata:
name: r2
---
apiVersion: example.com/v1
kind: Three
metadata:
name: r3
---
apiVersion: example.com/v1
kind: Four
metadata:
name: r4
`,
f: CloneOddsFilter{},
// output has four, but half are newly created nodes.
expected: `
apiVersion: example.com/v1
kind: One
metadata:
name: r1
---
apiVersion: example.com/v1
kind: One
metadata:
name: r1Clone
---
apiVersion: example.com/v1
kind: Three
metadata:
name: r3
---
apiVersion: example.com/v1
kind: Three
metadata:
name: r3Clone
`,
},
}
for name := range tests {
tc := tests[name]
t.Run(name, func(t *testing.T) {
m, err := rmF.NewResMapFromBytes([]byte(tc.input))
assert.NoError(t, err)
assert.NoError(t, m.ApplyFilter(tc.f))
kusttest_test.AssertActualEqualsExpectedWithTweak(
t, m, nil, tc.expected)
})
}
}
func TestApplySmPatch_Deletion(t *testing.T) {
target := `
apiVersion: apps/v1

View File

@@ -7,9 +7,9 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"sigs.k8s.io/kustomize/api/resid"
. "sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
func setupRMForPatchTargets(t *testing.T) ResMap {

View File

@@ -13,15 +13,21 @@ import (
"sigs.k8s.io/kustomize/api/internal/generators"
"sigs.k8s.io/kustomize/api/internal/kusterr"
"sigs.k8s.io/kustomize/api/konfig"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
// Factory makes instances of Resource.
type Factory struct {
hasher ifc.KustHasher
// When set to true, IncludeLocalConfigs indicates
// that Factory should include resources with the
// annotation 'config.kubernetes.io/local-config'.
// By default these resources are ignored.
IncludeLocalConfigs bool
}
// NewFactory makes an instance of Factory.
@@ -69,7 +75,7 @@ func (rf *Factory) makeOne(rn *yaml.RNode, o *types.GenArgs) *Resource {
if o == nil {
o = types.NewGenArgs(nil)
}
return &Resource{node: rn, options: o}
return &Resource{RNode: *rn, options: o}
}
// SliceFromPatches returns a slice of resources given a patch path
@@ -221,13 +227,15 @@ func (rf *Factory) shouldIgnore(n *yaml.RNode) (bool, error) {
if n.IsNilOrEmpty() {
return true, nil
}
md, err := n.GetValidatedMetadata()
if err != nil {
return true, err
}
_, ignore := md.ObjectMeta.Annotations[konfig.IgnoredByKustomizeAnnotation]
if ignore {
return true, nil
if !rf.IncludeLocalConfigs {
md, err := n.GetValidatedMetadata()
if err != nil {
return true, err
}
_, ignore := md.ObjectMeta.Annotations[konfig.IgnoredByKustomizeAnnotation]
if ignore {
return true, nil
}
}
if foundNil, path := n.HasNilEntryInList(); foundNil {
return true, fmt.Errorf("empty item at %v in object %v", path, n)

View File

@@ -3,7 +3,7 @@
package resource
import "sigs.k8s.io/kustomize/api/resid"
import "sigs.k8s.io/kustomize/kyaml/resid"
type IdSet struct {
ids map[resid.ResId]bool

View File

@@ -7,15 +7,14 @@ import (
"errors"
"fmt"
"log"
"reflect"
"strings"
"sigs.k8s.io/kustomize/api/filters/patchstrategicmerge"
"sigs.k8s.io/kustomize/api/ifc"
"sigs.k8s.io/kustomize/api/konfig"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/resid"
kyaml "sigs.k8s.io/kustomize/kyaml/yaml"
"sigs.k8s.io/yaml"
)
@@ -23,8 +22,7 @@ import (
// Resource is an RNode, representing a Kubernetes Resource Model object,
// paired with metadata used by kustomize.
type Resource struct {
// TODO: Inline RNode, dropping complexity. Resource is just a decorator.
node *kyaml.RNode
kyaml.RNode
options *types.GenArgs
refBy []resid.ResId
refVarNames []string
@@ -41,6 +39,7 @@ const (
// and kinds of their targets
buildAnnotationAllowNameChange = konfig.ConfigAnnoDomain + "/allowNameChange"
buildAnnotationAllowKindChange = konfig.ConfigAnnoDomain + "/allowKindChange"
allowed = "allowed"
)
var buildAnnotations = []string{
@@ -53,147 +52,21 @@ var buildAnnotations = []string{
buildAnnotationAllowKindChange,
}
func (r *Resource) AsRNode() *kyaml.RNode {
return r.node.Copy()
}
func (r *Resource) Node() *kyaml.RNode {
return r.node
}
func (r *Resource) ResetPrimaryData(incoming *Resource) {
r.node = incoming.node.Copy()
}
func (r *Resource) GetAnnotations() map[string]string {
annotations, err := r.node.GetAnnotations()
if err != nil || annotations == nil {
return make(map[string]string)
}
return annotations
}
func (r *Resource) GetFieldValue(f string) (interface{}, error) {
//nolint:staticcheck
return r.node.GetFieldValue(f)
}
func (r *Resource) GetDataMap() map[string]string {
return r.node.GetDataMap()
}
func (r *Resource) GetBinaryDataMap() map[string]string {
return r.node.GetBinaryDataMap()
func (r *Resource) ResetRNode(incoming *Resource) {
r.RNode = *incoming.Copy()
}
func (r *Resource) GetGvk() resid.Gvk {
meta, err := r.node.GetMeta()
if err != nil {
return resid.GvkFromString("")
}
g, v := resid.ParseGroupVersion(meta.APIVersion)
return resid.Gvk{Group: g, Version: v, Kind: meta.Kind}
return resid.GvkFromNode(&r.RNode)
}
func (r *Resource) Hash(h ifc.KustHasher) (string, error) {
return h.Hash(r.node)
}
func (r *Resource) GetKind() string {
return r.node.GetKind()
}
func (r *Resource) GetLabels() map[string]string {
l, err := r.node.GetLabels()
if err != nil {
return map[string]string{}
}
return l
}
func (r *Resource) GetName() string {
return r.node.GetName()
}
func (r *Resource) GetSlice(p string) ([]interface{}, error) {
//nolint:staticcheck
return r.node.GetSlice(p)
}
func (r *Resource) GetString(p string) (string, error) {
//nolint:staticcheck
return r.node.GetString(p)
}
func (r *Resource) IsEmpty() bool {
return r.node.IsNilOrEmpty()
}
func (r *Resource) Map() (map[string]interface{}, error) {
return r.node.Map()
}
func (r *Resource) MarshalJSON() ([]byte, error) {
return r.node.MarshalJSON()
}
func (r *Resource) MatchesLabelSelector(selector string) (bool, error) {
return r.node.MatchesLabelSelector(selector)
}
func (r *Resource) MatchesAnnotationSelector(selector string) (bool, error) {
return r.node.MatchesAnnotationSelector(selector)
}
func (r *Resource) SetAnnotations(m map[string]string) {
if len(m) == 0 {
// Force field erasure.
r.node.SetAnnotations(nil)
return
}
r.node.SetAnnotations(m)
}
func (r *Resource) SetDataMap(m map[string]string) {
r.node.SetDataMap(m)
}
func (r *Resource) SetBinaryDataMap(m map[string]string) {
r.node.SetBinaryDataMap(m)
return h.Hash(&r.RNode)
}
func (r *Resource) SetGvk(gvk resid.Gvk) {
r.node.SetMapField(
kyaml.NewScalarRNode(gvk.Kind), kyaml.KindField)
r.node.SetMapField(
kyaml.NewScalarRNode(gvk.ApiVersion()), kyaml.APIVersionField)
}
func (r *Resource) SetLabels(m map[string]string) {
if len(m) == 0 {
// Force field erasure.
r.node.SetLabels(nil)
return
}
r.node.SetLabels(m)
}
func (r *Resource) SetName(n string) {
r.node.SetName(n)
}
func (r *Resource) SetNamespace(n string) {
r.node.SetNamespace(n)
}
func (r *Resource) SetKind(k string) {
gvk := r.GetGvk()
gvk.Kind = k
r.SetGvk(gvk)
}
func (r *Resource) UnmarshalJSON(s []byte) error {
return r.node.UnmarshalJSON(s)
r.SetKind(gvk.Kind)
r.SetApiVersion(gvk.ApiVersion())
}
// ResCtx is an interface describing the contextual added
@@ -213,24 +86,36 @@ type ResCtxMatcher func(ResCtx) bool
// DeepCopy returns a new copy of resource
func (r *Resource) DeepCopy() *Resource {
rc := &Resource{
node: r.node.Copy(),
RNode: *r.Copy(),
}
rc.copyOtherFields(r)
rc.copyKustomizeSpecificFields(r)
return rc
}
// CopyMergeMetaDataFields copies everything but the non-metadata in
// CopyMergeMetaDataFieldsFrom copies everything but the non-metadata in
// the resource.
func (r *Resource) CopyMergeMetaDataFieldsFrom(other *Resource) {
r.SetLabels(mergeStringMaps(other.GetLabels(), r.GetLabels()))
r.SetAnnotations(
mergeStringMaps(other.GetAnnotations(), r.GetAnnotations()))
r.SetName(other.GetName())
r.SetNamespace(other.GetNamespace())
r.copyOtherFields(other)
// TODO: move to RNode, use GetMeta to improve performance.
// Must remove the kustomize bit at the end.
func (r *Resource) CopyMergeMetaDataFieldsFrom(other *Resource) error {
if err := r.SetLabels(
mergeStringMaps(other.GetLabels(), r.GetLabels())); err != nil {
return fmt.Errorf("copyMerge cannot set labels - %w", err)
}
if err := r.SetAnnotations(
mergeStringMaps(other.GetAnnotations(), r.GetAnnotations())); err != nil {
return fmt.Errorf("copyMerge cannot set annotations - %w", err)
}
if err := r.SetName(other.GetName()); err != nil {
return fmt.Errorf("copyMerge cannot set name - %w", err)
}
if err := r.SetNamespace(other.GetNamespace()); err != nil {
return fmt.Errorf("copyMerge cannot set namespace - %w", err)
}
r.copyKustomizeSpecificFields(other)
return nil
}
func (r *Resource) copyOtherFields(other *Resource) {
func (r *Resource) copyKustomizeSpecificFields(other *Resource) {
r.options = other.options
r.refBy = other.copyRefBy()
r.refVarNames = copyStringSlice(other.refVarNames)
@@ -286,12 +171,6 @@ func (r *Resource) ReferencesEqual(other *Resource) bool {
return len(setSelf) == len(setOther)
}
// NodeEqual returns true if the resource's nodes are
// equal, ignoring ancillary information like genargs, refby, etc.
func (r *Resource) NodeEqual(o *Resource) bool {
return reflect.DeepEqual(r.node, o.node)
}
func (r *Resource) copyRefBy() []resid.ResId {
if r.refBy == nil {
return nil
@@ -330,7 +209,9 @@ func (r *Resource) appendCsvAnnotation(name, value string) {
} else {
annotations[name] = value
}
r.SetAnnotations(annotations)
if err := r.SetAnnotations(annotations); err != nil {
panic(err)
}
}
func SameEndingSubarray(shortest, longest []string) bool {
@@ -385,7 +266,9 @@ func (r *Resource) RemoveBuildAnnotations() {
for _, a := range buildAnnotations {
delete(annotations, a)
}
r.SetAnnotations(annotations)
if err := r.SetAnnotations(annotations); err != nil {
panic(err)
}
}
func (r *Resource) setPreviousId(ns string, n string, k string) *Resource {
@@ -395,32 +278,34 @@ func (r *Resource) setPreviousId(ns string, n string, k string) *Resource {
return r
}
func (r *Resource) SetAllowNameChange(value string) {
// AllowNameChange allows name changes to the resource.
func (r *Resource) AllowNameChange() {
annotations := r.GetAnnotations()
annotations[buildAnnotationAllowNameChange] = value
r.SetAnnotations(annotations)
annotations[buildAnnotationAllowNameChange] = allowed
if err := r.SetAnnotations(annotations); err != nil {
panic(err)
}
}
func (r *Resource) NameChangeAllowed() bool {
annotations := r.GetAnnotations()
if allowed, set := annotations[buildAnnotationAllowNameChange]; set && allowed == "true" {
return true
}
return false
v, ok := annotations[buildAnnotationAllowNameChange]
return ok && v == allowed
}
func (r *Resource) SetAllowKindChange(value string) {
// AllowKindChange allows kind changes to the resource.
func (r *Resource) AllowKindChange() {
annotations := r.GetAnnotations()
annotations[buildAnnotationAllowKindChange] = value
r.SetAnnotations(annotations)
annotations[buildAnnotationAllowKindChange] = allowed
if err := r.SetAnnotations(annotations); err != nil {
panic(err)
}
}
func (r *Resource) KindChangeAllowed() bool {
annotations := r.GetAnnotations()
if allowed, set := annotations[buildAnnotationAllowKindChange]; set && allowed == "true" {
return true
}
return false
v, ok := annotations[buildAnnotationAllowKindChange]
return ok && v == allowed
}
// String returns resource as JSON.
@@ -467,13 +352,6 @@ func (r *Resource) NeedHashSuffix() bool {
return r.options != nil && r.options.ShouldAddHashSuffixToName()
}
// GetNamespace returns the namespace the resource thinks it's in.
func (r *Resource) GetNamespace() string {
namespace, _ := r.GetString("metadata.namespace")
// if err, namespace is empty, so no need to check.
return namespace
}
// OrgId returns the original, immutable ResId for the resource.
// This doesn't have to be unique in a ResMap.
func (r *Resource) OrgId() resid.ResId {
@@ -554,11 +432,11 @@ func (r *Resource) ApplySmPatch(patch *Resource) error {
r.StorePreviousId()
}
if err := r.ApplyFilter(patchstrategicmerge.Filter{
Patch: patch.node,
Patch: &patch.RNode,
}); err != nil {
return err
}
if r.IsEmpty() {
if r.IsNilOrEmpty() {
return nil
}
if !patch.KindChangeAllowed() {
@@ -572,10 +450,11 @@ func (r *Resource) ApplySmPatch(patch *Resource) error {
}
func (r *Resource) ApplyFilter(f kio.Filter) error {
l, err := f.Filter([]*kyaml.RNode{r.node})
l, err := f.Filter([]*kyaml.RNode{&r.RNode})
if len(l) == 0 {
// The node was deleted. The following makes r.IsEmpty() true.
r.node = nil
// The node was deleted, which means the entire resource
// must be deleted. Signal that via the following:
r.SetYNode(nil)
}
return err
}

View File

@@ -10,9 +10,9 @@ import (
"github.com/stretchr/testify/assert"
"sigs.k8s.io/kustomize/api/provider"
"sigs.k8s.io/kustomize/api/resid"
. "sigs.k8s.io/kustomize/api/resource"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
var factory = provider.NewDefaultDepProvider().GetResourceFactory()
@@ -87,11 +87,13 @@ func TestResourceId(t *testing.T) {
{
in: testConfigMap,
id: resid.NewResIdWithNamespace(
resid.Gvk{Version: "v1", Kind: "ConfigMap"}, "winnie", "hundred-acre-wood"),
resid.NewGvk("", "v1", "ConfigMap"),
"winnie", "hundred-acre-wood"),
},
{
in: testDeployment,
id: resid.NewResId(resid.Gvk{Group: "apps", Version: "v1", Kind: "Deployment"}, "pooh"),
id: resid.NewResId(
resid.NewGvk("apps", "v1", "Deployment"), "pooh"),
},
}
for _, test := range tests {

View File

@@ -102,7 +102,10 @@ func (th *HarnessEnhanced) MkDir(path string) string {
func (th *HarnessEnhanced) Reset() {
if th.shouldWipeLdrRoot {
if !strings.HasPrefix(th.ldr.Root(), os.TempDir()) {
root, _ := filepath.EvalSymlinks(th.ldr.Root())
tmpdir, _ := filepath.EvalSymlinks(os.TempDir())
if !strings.HasPrefix(root, tmpdir) {
// sanity check.
panic("something strange about th.ldr.Root() = " + th.ldr.Root())
}

View File

@@ -19,8 +19,15 @@ func assertActualEqualsExpectedWithTweak(
ht hasGetT,
m resmap.ResMap,
tweaker func([]byte) []byte, expected string) {
AssertActualEqualsExpectedWithTweak(ht.GetT(), m, tweaker, expected)
}
func AssertActualEqualsExpectedWithTweak(
t *testing.T,
m resmap.ResMap,
tweaker func([]byte) []byte, expected string) {
if m == nil {
ht.GetT().Fatalf("Map should not be nil.")
t.Fatalf("Map should not be nil.")
}
// Ignore leading linefeed in expected value
// to ease readability of tests.
@@ -29,13 +36,13 @@ func assertActualEqualsExpectedWithTweak(
}
actual, err := m.AsYaml()
if err != nil {
ht.GetT().Fatalf("Unexpected err: %v", err)
t.Fatalf("Unexpected err: %v", err)
}
if tweaker != nil {
actual = tweaker(actual)
}
if string(actual) != expected {
reportDiffAndFail(ht.GetT(), actual, expected)
reportDiffAndFail(t, actual, expected)
}
}

View File

@@ -7,9 +7,9 @@ import (
"testing"
"sigs.k8s.io/kustomize/api/provider"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/resource"
"sigs.k8s.io/kustomize/kyaml/resid"
)
// Builds ResMaps for tests, with test-aware error handling.

View File

@@ -6,7 +6,7 @@ package types
import (
"fmt"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/kyaml/resid"
)
// FieldSpec completely specifies a kustomizable field in a k8s API object.

View File

@@ -9,8 +9,8 @@ import (
"strings"
"testing"
"sigs.k8s.io/kustomize/api/resid"
. "sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
var mergeTests = []struct {

View File

@@ -39,7 +39,7 @@ func (g *GenArgs) ShouldAddHashSuffixToName() bool {
// Behavior returns Behavior field of GeneratorArgs
func (g *GenArgs) Behavior() GenerationBehavior {
if g.args == nil {
if g == nil || g.args == nil {
return BehaviorUnspecified
}
return NewGenerationBehavior(g.args.Behavior)

View File

@@ -51,6 +51,10 @@ type HelmChart struct {
// If omitted, the flag --generate-name is passed to 'helm template'.
ReleaseName string `json:"releaseName,omitempty" yaml:"releaseName,omitempty"`
// Namespace set the target namespace for a release. It is .Release.Namespace
// in the helm template
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
// ValuesFile is local file path to a values file to use _instead of_
// the default values that accompanied the chart.
// The default values are in '{ChartHome}/{Name}/values.yaml'.
@@ -64,6 +68,10 @@ type HelmChart struct {
// Legal values: 'merge', 'override', 'replace'.
// Defaults to 'override'.
ValuesMerge string `json:"valuesMerge,omitempty" yaml:"valuesMerge,omitempty"`
// IncludeCRDs specifies if Helm should also generate CustomResourceDefinitions.
// Defaults to 'false'.
IncludeCRDs bool `json:"includeCRDs,omitempty" yaml:"includeCRDs,omitempty"`
}
// HelmChartArgs contains arguments to helm.

View File

@@ -0,0 +1,36 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package types
type Cloud string
const GKE Cloud = "gke"
// IAMPolicyGeneratorArgs contains arguments to generate a GKE service account resource.
type IAMPolicyGeneratorArgs struct {
// which cloud provider to generate for (e.g. "gke")
Cloud `json:"cloud" yaml:"cloud"`
// information about the kubernetes cluster for this object
KubernetesService `json:"kubernetesService" yaml:"kubernetesService"`
// information about the service account and project
ServiceAccount `json:"serviceAccount" yaml:"serviceAccount"`
}
type KubernetesService struct {
// the name used for the Kubernetes service account
Name string `json:"name" yaml:"name"`
// the name of the Kubernetes namespace for this object
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
}
type ServiceAccount struct {
// the name of the new cloud provider service account
Name string `json:"name" yaml:"name"`
// The ID of the project
ProjectId string `json:"projectId" yaml:"projectId"`
}

View File

@@ -3,8 +3,8 @@ package types_test
import (
"testing"
"sigs.k8s.io/kustomize/api/resid"
. "sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
func TestPatchEquals(t *testing.T) {

View File

@@ -7,7 +7,7 @@ import (
"fmt"
"strings"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/kyaml/resid"
)
const DefaultReplacementFieldPath = "metadata.name"
@@ -28,10 +28,10 @@ type SourceSelector struct {
resid.ResId `json:",inline,omitempty" yaml:",inline,omitempty"`
// Structured field path expected in the allowed object.
FieldPath string `json:"fieldPath" yaml:"fieldPath"`
FieldPath string `json:"fieldPath,omitempty" yaml:"fieldPath,omitempty"`
// Used to refine the interpretation of the field.
Options *FieldOptions `json:"options" yaml:"options"`
Options *FieldOptions `json:"options,omitempty" yaml:"options,omitempty"`
}
func (s *SourceSelector) String() string {
@@ -54,34 +54,34 @@ type TargetSelector struct {
Select *Selector `json:"select" yaml:"select"`
// From the allowed set, remove objects that match this.
Reject []*Selector `json:"reject" yaml:"reject"`
Reject []*Selector `json:"reject,omitempty" yaml:"reject,omitempty"`
// Structured field paths expected in each allowed object.
FieldPaths []string `json:"fieldPaths" yaml:"fieldPaths"`
FieldPaths []string `json:"fieldPaths,omitempty" yaml:"fieldPaths,omitempty"`
// Used to refine the interpretation of the field.
Options *FieldOptions `json:"options" yaml:"options"`
Options *FieldOptions `json:"options,omitempty" yaml:"options,omitempty"`
}
// FieldOptions refine the interpretation of FieldPaths.
type FieldOptions struct {
// Used to split/join the field.
Delimiter string `json:"delimiter" yaml:"delimiter"`
Delimiter string `json:"delimiter,omitempty" yaml:"delimiter,omitempty"`
// Which position in the split to consider.
Index int `json:"index" yaml:"index"`
Index int `json:"index,omitempty" yaml:"index,omitempty"`
// TODO (#3492): Implement use of this option
// None, Base64, URL, Hex, etc
Encoding string `json:"encoding" yaml:"encoding"`
Encoding string `json:"encoding,omitempty" yaml:"encoding,omitempty"`
// If field missing, add it.
Create bool `json:"create" yaml:"create"`
Create bool `json:"create,omitempty" yaml:"create,omitempty"`
}
func (fo *FieldOptions) String() string {
if fo == nil || fo.Delimiter == "" {
if fo == nil || (fo.Delimiter == "" && !fo.Create) {
return ""
}
return fmt.Sprintf("%s(%d)", fo.Delimiter, fo.Index)
return fmt.Sprintf("%s(%d), create=%t", fo.Delimiter, fo.Index, fo.Create)
}

View File

@@ -7,7 +7,7 @@ import (
"fmt"
"regexp"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/kyaml/resid"
)
// Selector specifies a set of resources.
@@ -28,6 +28,10 @@ type Selector struct {
LabelSelector string `json:"labelSelector,omitempty" yaml:"labelSelector,omitempty"`
}
func (s *Selector) Copy() Selector {
return *s
}
func (s *Selector) String() string {
return fmt.Sprintf(
"%s:a=%s:l=%s", s.ResId, s.AnnotationSelector, s.LabelSelector)

View File

@@ -3,8 +3,8 @@ package types_test
import (
"testing"
"sigs.k8s.io/kustomize/api/resid"
. "sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
)
func TestSelectorRegexMatchGvk(t *testing.T) {

View File

@@ -9,7 +9,7 @@ import (
"sort"
"strings"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/kyaml/resid"
)
// Var represents a variable whose value will be sourced

View File

@@ -9,7 +9,7 @@ import (
"testing"
"gopkg.in/yaml.v2"
"sigs.k8s.io/kustomize/api/resid"
"sigs.k8s.io/kustomize/kyaml/resid"
)
func TestGVK(t *testing.T) {

View File

@@ -17,17 +17,18 @@ func NewCommand() *cobra.Command {
DisableFlagsInUseLine: true,
ValidArgs: []string{"bash", "zsh", "fish", "powershell"},
Args: cobra.ExactValidArgs(1),
Run: func(cmd *cobra.Command, args []string) {
RunE: func(cmd *cobra.Command, args []string) error {
switch args[0] {
case "bash":
cmd.Root().GenBashCompletion(os.Stdout)
return cmd.Root().GenBashCompletion(os.Stdout)
case "zsh":
cmd.Root().GenZshCompletion(os.Stdout)
return cmd.Root().GenZshCompletion(os.Stdout)
case "fish":
cmd.Root().GenFishCompletion(os.Stdout, true)
return cmd.Root().GenFishCompletion(os.Stdout, true)
case "powershell":
cmd.Root().GenPowerShellCompletion(os.Stdout)
return cmd.Root().GenPowerShellCompletion(os.Stdout)
}
return nil
},
}
}

View File

@@ -1,65 +0,0 @@
# Configuration IO API Semantics
Resource Configuration may be read / written from / to sources such as directories,
stdin|out or network. Tools may be composed using pipes such that the tools writing
Resource Configuration may be a different tool from the one that read the configuration.
In order for tools to be composed in this way, while preserving origin information --
such as the original file, index, etc.:
Tools **SHOULD** insert the following annotations when reading from sources,
and **SHOULD** delete the annotations when writing to sinks.
### `config.kubernetes.io/path`
Records the slash-delimited, OS-agnostic, relative file path to a Resource.
This annotation **SHOULD** be set when reading Resources from files.
It **SHOULD** be unset when writing Resources to files.
When writing Resources to a directory, the Resource **SHOULD** be written to the corresponding
path relative to that directory.
Example:
```yaml
metadata:
annotations:
config.kubernetes.io/path: "relative/file/path.yaml"
```
### `config.kubernetes.io/index`
Records the index of a Resource in file. In a multi-object YAML file, Resources are separated
by three dashes (`---`), and the index represents the position of the Resource starting from zero.
This annotation **SHOULD** be set when reading Resources from files.
It **SHOULD** be unset when writing Resources to files.
When writing multiple Resources to the same file, the Resource **SHOULD** be written in the
relative order matching the index.
When this annotation is not specified, it implies a value of `0`.
Example:
```yaml
metadata:
annotations:
config.kubernetes.io/path: "relative/file/path.yaml"
config.kubernetes.io/index: 2
```
This represents the third Resource in the file.
### `config.kubernetes.io/local-config`
`config.kubernetes.io/local-config` declares that the configuration is to local tools
rather than a remote Resource. e.g. The `Kustomization` config in a `kustomization.yaml`
**SHOULD** contain this annotation so that tools know it is not intended to be sent to
the Kubernetes api server.
Example:
```yaml
metadata:
annotations:
config.kubernetes.io/local-config: "true"
```

View File

@@ -1,13 +1,18 @@
# Configuration Functions Specification
# KRM Functions Specification
_apiVersion: v1_
## Overview
This document specifies a standard for client-side functions that operate on
Kubernetes declarative configurations. This standard enables creating
small, interoperable, and language-independent executable programs packaged as
containers that can be chained together as part of a configuration management pipeline.
The end result of such a pipeline are fully rendered configurations that can then be
applied to a control plane (e.g. Using kubectl apply for Kubernetes control plane).
As such, although this document references Kubernetes Resource Model and API conventions,
it is completely decoupled from Kubernetes API machinery and does not depend on any
Kubernetes declarative configurations referred to as _KRM Functions_. This
standard enables creating small, interoperable, and language-independent
executable programs packaged as containers that can be chained together as part
of a configuration management pipeline. The end result of such a pipeline are
fully rendered configurations that can then be applied to a control plane (e.g.
Using kubectl apply for Kubernetes control plane). As such, although this
document references Kubernetes Resource Model and API conventions, it is
completely decoupled from Kubernetes API machinery and does not depend on any
in-cluster components.
This document references terms described in [Kubernetes API Conventions][1].
@@ -18,168 +23,359 @@ interpreted as described in [RFC 2119][2].
## Use Cases
_Configuration functions_ enable shift-left practices (client-side) through:
KRM functions enable shift-left practices (client-side) through:
- Pre-commit / delivery validation and linting of configuration
- e.g. Fail if any containers don't have PodSecurityPolicy or CPU / Memory limits
- Implementation of abstractions as client actuated APIs (e.g. templating)
- e.g. Create a client-side _"CRD"_ for generating configuration checked into git
- Aspect Orient configuration / Injection of cross-cutting configuration
- e.g. T-Shirt size containers by annotating Resources with `small`, `medium`, `large`
and inject the cpu and memory resources into containers accordingly.
- e.g. Inject `init` and `side-car` containers into Resources based off of Resource
Type, annotations, etc.
- e.g. Fail if any containers don't have PodSecurityPolicy or CPU / Memory
limits
- Implementation of abstractions as client actuated APIs
- e.g. Create a client-side _"CRD"_ for generating configuration checked into
git
- Injection of cross-cutting configuration
- e.g. T-Shirt size containers by annotating resources with `small`, `medium`,
`large` and inject the cpu and memory resources into containers accordingly.
- e.g. Inject `init` and `side-car` containers into resources based off of
resource type, annotations, etc.
Performing these on the client rather than the server enables:
- Configuration to be reviewed prior to being sent to the API server
- Configuration to be validated as part of the CI/CD pipeline
- Configuration for Resources to validated holistically rather than individually
per-Resource
- Configuration for resources to validated holistically rather than individually
per-resource
- e.g. ensure the `Service.selector` and `Deployment.spec.template` labels
match.
- e.g. MutatingWebHooks are scoped to a single Resource instance at a time.
- e.g. MutatingWebHooks are scoped to a single resource instance at a time.
- Low-level tweaks to the output of high-level abstractions
- e.g. add an `init container` to a client _"CRD"_ Resource after it was generated.
- e.g. add an `init container` to a client _"CRD"_ resource after it was
generated.
- Composition and layering of multiple functions together
- Compose generation, injection, validation together
## Spec
## Definitions
### Input Type
- **function:** A containerized program conforming to the spec described in this
document.
- **orchestrator:** A program that invokes the function container, passing
arguments and processing its output.
A function MUST accept as input a single [Kubernetes List type][3].
The `items` field in the input will contain a sequence of [Object types][3].
A function MAY not support [Simple types][3] and List types.
## Interface
An example using `v1/ConfigMapList` as input:
The inter-process communication between the orchestrator and a function works as
follows:
1. Orchestrator runs the function container and provides the input on `stdin`.
The input is a Kubernetes object of kind `ResourceList` as described below.
2. Function reads the input from `stdin`, performs computations, and provides
the output as a `ResourceList` to `stdout`. The function MAY also emit
non-structured error message on `stderr`.
3. Orchestrator uses the `stdout`, `stderr`, and the exit code of the function
as it sees fit following to the semantics described below.
### Schema
A function MUST accept input from `stdin` and MUST output to `stdout` a
Kubernetes object of kind `ResourceList` with the following OpenAPI schema:
```yaml
apiVersion: v1
kind: ConfigMapList
items:
- apiVersion: v1
kind: ConfigMap
metadata:
name: config1
data:
p1: v1
p2: v2
- apiVersion: v1
kind: ConfigMap
metadata:
name: config2
swagger: "2.0"
info:
title: KRM Functions Specification (ResourceList)
version: v1
definitions:
ResourceList:
type: object
description: ResourceList is the input/output wire format for KRM functions.
x-kubernetes-group-version-kind:
- group: config.kubernetes.io
kind: ResourceList
version: v1
- group: config.kubernetes.io
kind: ResourceList
version: v1beta1
required:
- items
properties:
apiVersion:
description: apiVersion of ResourceList
type: string
kind:
description: kind of ResourceList i.e. `ResourceList`
type: string
items:
type: array
description: |
[input/output]
Items is a list of Kubernetes objects:
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#types-kinds).
A function will read this field in the input ResourceList and populate
this field in the output ResourceList.
items:
type: object
functionConfig:
type: object
description: |
[input]
FunctionConfig is an optional Kubernetes object for passing arguments to a
function invocation.
results:
type: array
description: |
[output]
Results is an optional list that can be used by function to emit results
for observability and debugging purposes.
items:
"$ref": "#/definitions/Result"
Result:
type: object
required:
- message
properties:
message:
type: string
description: Message is a human readable message.
severity:
type: string
enum:
- error
- warning
- info
default: error
description: |
Severity is the severity of a result:
"error": indicates an error result.
"warning": indicates a warning result.
"info": indicates an informational result.
resourceRef:
type: object
description: |
ResourceRef is the metadata for referencing a Kubernetes object
associated with a result.
required:
- apiVersion
- kind
- name
properties:
apiVersion:
description:
APIVersion refers to the `apiVersion` field of the object
manifest.
type: string
kind:
description: Kind refers to the `kind` field of the object.
type: string
namespace:
description:
Namespace refers to the `metadata.namespace` field of the object
manifest.
type: string
name:
description:
Name refers to the `metadata.name` field of the object manifest.
type: string
field:
type: object
description: |
Field is the reference to a field in the object.
If defined, `ResourceRef` must also be provided.
required:
- path
properties:
path:
type: string
description: |
Path is the JSON path of the field
e.g. `spec.template.spec.containers[3].resources.limits.cpu`
currentValue:
description: |
CurrrentValue is the current value of the field.
Can be any value - string, number, boolean, array or object.
proposedValue:
description: |
PropposedValue is the proposed value of the field to fix an issue.
Can be any value - string, number, boolean, array or object.
file:
type: object
description: File references a file containing the resource.
required:
- path
properties:
path:
type: string
description: |
Path is the OS agnostic, slash-delimited, relative path.
e.g. `some-dir/some-file.yaml`.
index:
type: number
default: 0
description: Index of the object in a multi-object YAML file.
tags:
type: object
additionalProperties:
type: string
description: |
Tags is an unstructured key value map stored with a result that may be set
by external tools to store and retrieve arbitrary metadata.
paths: {}
```
An example using `v1/List` as input:
#### Examples
The following is an example input, where the custom resource of kind
`FulfillmentCenter` is provided as `functionConfig`. The function will operate
on one resource of kind `Service`.
```yaml
apiVersion: v1
kind: List
items:
- apiVersion: foo-corp.com/v1
kind: FulfillmentCenter
metadata:
name: staging
address: "100 Main St."
- apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: namespace-reader
rules:
- resources:
- namespaces
apiGroups:
- ""
verbs:
- get
- watch
- list
```
In addition, a function MUST accept as input a List of kind `ResourceList` where the
`functionConfig` field, if present, will contain the invocation-specific configuration passed to the function
by the orchestrator.
Functions MAY consider this field optional so that they can be triggered in an ad-hoc fashion.
An example using `config.kubernetes.io/v1beta1/ResourceList` as input:
```yaml
apiVersion: config.kubernetes.io/v1beta1
apiVersion: config.kubernetes.io/v1
kind: ResourceList
functionConfig:
apiVersion: foo-corp.com/v1
kind: FulfillmentCenter
metadata:
name: staging
metadata:
annotations:
config.kubernetes.io/function: |
container:
image: gcr.io/example/foo:v1.0.0
spec:
address: "100 Main St."
items:
- apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
- apiVersion: v1
kind: Service
metadata:
name: namespace-reader
rules:
- resources:
- namespaces
apiGroups:
- ""
verbs:
- get
- watch
- list
name: wordpress
labels:
app: wordpress
annotations:
config.kubernetes.io/index: "0"
config.kubernetes.io/path: "service.yaml"
spec: # Example comment
type: LoadBalancer
selector:
app: wordpress
tier: frontend
ports:
- protocol: TCP
port: 80
```
Here `FulfillmentCenter` kind with name `staging` is passed as the invocation-specific configuration
to the function.
The following is an example output containing one result representing a
validation error:
### Output Type
A functions output MUST be the same as the input specification above
-- i.e. `ResourceList` or `List`.
This is necessary to enable chaining two or more functions together in a pipeline.
The serialization format of the output SHOULD match that of its input on each invocation
-- e.g. if the input was a `ResourceList`, the output should also be a `ResourceList`.
```yaml
apiVersion: config.kubernetes.io/v1
kind: ResourceList
items:
- apiVersion: v1
kind: Service
metadata:
name: wordpress
labels:
app: wordpress
annotations:
config.kubernetes.io/index: "0"
config.kubernetes.io/path: "service.yaml"
spec: # Example comment
type: LoadBalancer
selector:
app: wordpress
tier: frontend
ports:
- protocol: TCP
port: 80
results:
- message: "Invalid type. Expected: integer, given: string"
severity: error
resourceRef:
apiVersion: v1
kind: Service
name: wordpress
field:
path: spec.ports.0.port
file:
path: service.yaml
```
### Serialization Format
A function MUST support YAML as a serialization format for the input and output.
A function MUST use utf8 encoding (as YAML is a superset of JSON, JSON will also be supported
by any conforming function).
### Operations
A function MAY Create, Update, or Delete any number of items in the `items` field and output the
resultant list.
A function MAY modify annotations with prefix `config.kubernetes.io`, but must be careful about
doing so since theyre used for orchestration purposes and will likely impact subsequent functions
in the pipeline.
A function SHOULD preserve comments when input serialization format is YAML.
This allows for human authoring of configuration to coexist with changes made by functions.
A function MUST use utf8 encoding (as YAML is a superset of JSON, JSON will also
be supported by any conforming function).
### Containerization
A function MUST be implemented as a container.
A function container MUST be capable of running as a non-root user if it does not require
access to host filesystem or makes network calls.
A function container MUST be capable of running as a non-root user `nobody` if
it does not require access to host filesystem.
### stdin/stdout/stderr and Exit Codes
### stderr
A function MUST accept input from stdin and emit output to stdout.
Any non-structured error messages MUST be emitted to `stderr`. `stdout` is
reserved for `ResourceList` as described above.
Any error messages MUST be emitted to stderr.
### Exit Code
An exit code of zero indicates function execution was successful.
A non-zero exit code indicates a failure.
An exit code of zero indicates function execution was successful. A non-zero
exit code indicates a failure.
[1]: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md
### Operations
A function MAY Create, Update, or Delete any number of items in the `items`
field and output the resultant list in the corresponding `items` field of the
output.
A function SHOULD preserve comments when input serialization format is YAML.
This allows for human authoring of configuration to coexist with changes made by
functions.
### Annotations
The orchestrator annotates resources in the wire format with annotation prefix
`config.kubernetes.io`. These annotations are not persisted when the
orchestrator writes the resources to the filesystem. The orchestrator sets this
annotation when reading files from the local filesystem and removes the
annotation when writing the output of functions back to the filesystem.
In general, a function MUST NOT modify these annotations except the ones
explicitly listed below.
#### `config.kubernetes.io/path`
Records the slash-delimited, OS-agnostic, relative file path to a resource. The
path is relative to a fix location on the filesystem. Different orchestrator
implementations can choose different fixed points.
A function SHOULD NOT modify this annotation.
Example:
```yaml
metadata:
annotations:
config.kubernetes.io/path: "relative/file/path.yaml"
```
#### `config.kubernetes.io/index`
Records the index of a Resource in file. In a multi-object YAML file, resources
are separated by three dashes (`---`), and the index represents the position of
the Resource starting from zero. When this annotation is not specified, it
implies a value of `0`.
A function SHOULD NOT modify this annotation.
Example:
```yaml
metadata:
annotations:
config.kubernetes.io/path: "relative/file/path.yaml"
config.kubernetes.io/index: 2
```
This represents the third resource in the file.
[1]:
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md
[2]: https://tools.ietf.org/html/rfc2119
[3]: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#types-kinds
[3]:
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#types-kinds

View File

@@ -16,7 +16,9 @@ require (
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f // indirect
gopkg.in/inf.v0 v0.9.1
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e
sigs.k8s.io/kustomize/kyaml v0.10.18
sigs.k8s.io/kustomize/kyaml v0.10.20
)
replace gopkg.in/yaml.v3 => gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
replace sigs.k8s.io/kustomize/kyaml => ../../kyaml

View File

@@ -47,7 +47,6 @@ github.com/go-openapi/jsonreference v0.19.3/go.mod h1:rjx6GuL8TTa9VaixXglHmQmIL9
github.com/go-openapi/swag v0.19.5 h1:lTz6Ys4CmqqCQmZPBlbQENR1/GucA2bzYTE12Pw4tFY=
github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/gobuffalo/here v0.6.0/go.mod h1:wAG085dHOYqUpf+Ap+WOdrPTp5IYcDAs/x7PLa8Y5fM=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
@@ -98,7 +97,6 @@ github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN
github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mailru/easyjson v0.7.0 h1:aizVhC/NAAcKWb+5QsU1iNOZb4Yws5UO2I+aIprQITM=
github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs=
github.com/markbates/pkger v0.17.1/go.mod h1:0JoVlrol20BSywW79rN3kdFFsE5xYM+rSCQDXbLhiuI=
github.com/mattn/go-runewidth v0.0.7 h1:Ei8KR0497xHyKJPAv59M1dkC+rOZCMBJ+t3fZ+twI54=
github.com/mattn/go-runewidth v0.0.7/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
@@ -236,7 +234,6 @@ gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bl
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.7/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
@@ -247,7 +244,5 @@ k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8
k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE=
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e h1:KLHHjkdQFomZy8+06csTWZ0m1343QqxZhR2LJ1OxCYM=
k8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e/go.mod h1:vHXdDvt9+2spS2Rx9ql3I8tycm3H9FDfdUoIuKCefvw=
sigs.k8s.io/kustomize/kyaml v0.10.18 h1:Cuf4KiVULTttfo/2Vls2H9fA7eH8Xll1w6RgGdL+tR8=
sigs.k8s.io/kustomize/kyaml v0.10.18/go.mod h1:h94DSoDbmnN4BTc6VTX7tGNGXZy29rbPo+R4jGMvA8U=
sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw=
sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc=

View File

@@ -104,9 +104,9 @@ func (r *AnnotateRunner) ExecuteCmd(w io.Writer, pkgPath string) error {
return err
}
// print error message and continue if there are multiple packages to annotate
fmt.Fprintf(w, "%s\n", err.Error())
_, _ = fmt.Fprintf(w, "%s\n", err.Error())
} else {
fmt.Fprint(w, "added annotations in the package\n")
_, _ = fmt.Fprint(w, "added annotations in the package\n")
}
return nil
}

View File

@@ -87,8 +87,18 @@ metadata:
expectedFiles: func(d string) map[string]string {
return map[string]string{
"deployment.json": `
{"apiVersion": "apps/v1", "kind": "Deployment", "metadata": {"name": "foo", annotations: {
a-string-value: '', a-int-value: '0', a-bool-value: 'false'}}}
{
"apiVersion": "apps/v1",
"kind": "Deployment",
"metadata": {
"annotations": {
"a-bool-value": "false",
"a-int-value": "0",
"a-string-value": ""
},
"name": "foo"
}
}
`,
}
},

Some files were not shown because too many files have changed in this diff Show More