Compare commits

..

1 Commits

Author SHA1 Message Date
Jeff Regan
9f5697b154 Update OWNERS_ALIASES 2021-06-08 16:54:20 -07:00
12752 changed files with 982965 additions and 125568 deletions

View File

@@ -1,6 +1,7 @@
.github
docs
examples
functions
hack
site
travis

68
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,68 @@
---
name: Bug report
about: Create a report to help us improve
title: ""
labels:
- kind/bug
assignees: ""
---
<!--
Please read this page: https://kubernetes-sigs.github.io/kustomize/contributing/bugs/ before
filing a bug
-->
<!-- Feel free to skip the sections if they are not applicable. -->
**Describe the bug**
<!-- A clear and concise description of what the bug is. -->
**Files that can reproduce the issue**
<!--
We cannot figure out or fix the issue if we don't know how to reproduce. Please
provide a minimum set of files that can reproduce the issue. You can paste the
file contents here or provide a link to a tarball or git repo.
Example:
kustomization.yaml
```
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
...
```
resources.yaml
```
apiVersion: v1
kind: Deployment
...
```
...
-->
**Expected output**
<!-- What's the expected output? -->
**Actual output**
<!-- What's the actual output? -->
**Kustomize version**
<!-- Please use the latest version when it's possible. -->
**Platform**
<!-- Linux/macOS/Windows? -->
**Additional context**
<!-- Add any other context about the problem here. -->

View File

@@ -1,87 +0,0 @@
name: Bug report
description: File a bug report
labels: ["kind/bug"]
body:
- type: textarea
id: problem
attributes:
label: What happened?
description: |
Please provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner.
If this matter is security related, please disclose it privately via https://kubernetes.io/security
validations:
required: true
- type: textarea
id: expected
attributes:
label: What did you expect to happen?
validations:
required: true
- type: textarea
id: repro-files
attributes:
label: How can we reproduce it (as minimally and precisely as possible)?
description: Please provide a minimum set of files that can reproduce the issue. You can paste the file contents here or provide a link to a tarball or git repo. Even better, submit a tests case in the api/krusty package! For more information on how to do that, see https://kubectl.docs.kubernetes.io/contributing/kustomize/bugs/.
value: |
```yaml
# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- resources.yaml
```
```yaml
# resources.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: test-object
data:
placeholder: data
```
validations:
required: true
- type: textarea
id: expected-output
attributes:
label: Expected output
description: If you are able to provide reproduction files, please include the output you expect here.
value: |
```yaml
```
validations:
required: false
- type: textarea
id: actual-output
attributes:
label: Actual output
description: If you are able to provide reproduction files, please include the output they currently produce here.
value: |
```yaml
```
validations:
required: false
- type: input
id: kustomize-version
attributes:
label: Kustomize version
description: Please use the latest version whenever possible.
placeholder: What version of Kustomize are you using?
validations:
required: true
- type: dropdown
id: os
attributes:
label: Operating system
options:
- Linux
- MacOS
- Windows
- Other
validations:
required: false

1
.github/ISSUE_TEMPLATE/config.yaml vendored Normal file
View File

@@ -0,0 +1 @@
blank_issues_enabled: true

View File

@@ -1,6 +0,0 @@
contact_links:
- name: Support request
url: https://discuss.kubernetes.io
about: |
Please do not submit support requests or questions as issues.
Ask your question in the Kubernetes Community Forums, or in the #kustomize channel on Kubernetes Slack (https://slack.k8s.io).

View File

@@ -0,0 +1,26 @@
---
name: Feature request
about: Suggest an idea for this project
title: ""
labels:
- kind/feature
assignees: ""
---
<!-- Feel free to skip the sections if they are not applicable. -->
**Is your feature request related to a problem? Please describe.**
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
**Describe the solution you'd like**
<!-- A clear and concise description of what you want to happen. -->
**Describe alternatives you've considered**
<!-- A clear and concise description of any alternative solutions or features you've considered. -->
**Additional context**
<!-- Add any other context or screenshots about the feature request here. -->

View File

@@ -1,64 +0,0 @@
name: Feature request
description: Propose an enhancement to Kustomize
labels: kind/feature
body:
- type: markdown
attributes:
value: |
Small, straightforward enhancements can be proposed in regular GitHub issues using the template below. As a rule of thumb, the enhancement should be resolvable in a single PR that is at most size L. Anything more involved requires a mini (in-repo) enhancement proposal, and features with implications for kubectl require a full [KEP](https://github.com/kubernetes/enhancements).
For more information on the Kustomize enhancement process, see: https://github.com/kubernetes-sigs/kustomize/tree/master/proposals.
When in doubt, go ahead and fill out the template below; the maintainers will let you know if a KEP is required.
- type: checkboxes
attributes:
label: Eschewed features
description: Some features are out of scope for Kustomize because they are incompatible with its foundational design principles. Please review the [Eschewed Features](https://kubectl.docs.kubernetes.io/faq/kustomize/eschewedfeatures/) documentation before submitting your feature request.
options:
- label: This issue is not requesting templating, unstuctured edits, build-time side-effects from args or env vars, or any other eschewed feature.
required: true
- type: textarea
id: feature-description
attributes:
label: What would you like to have added?
validations:
required: true
- type: textarea
id: rationale
attributes:
label: Why is this needed?
validations:
required: true
- type: textarea
id: current-alternatives
attributes:
label: Can you accomplish the motivating task without this feature, and if so, how?
validations:
required: true
- type: textarea
id: design-alternatives
attributes:
label: What other solutions have you considered?
validations:
required: true
- type: textarea
id: additional-info
attributes:
label: Anything else we should know?
validations:
required: false
- type: checkboxes
attributes:
label: Feature ownership
description: The Kustomize project, like many areas of Kubernetes, currently lacks enough contributors to adequately respond to all proposals that have merit. Offering to build and support the feature yourself can help get traction for your request.
options:
- label: I am interested in contributing this feature myself! 🎉
required: false

9
.github/ISSUE_TEMPLATE/question.md vendored Normal file
View File

@@ -0,0 +1,9 @@
---
name: Question
about: Ask a question about the kustomize
title: "[Question]"
labels: ""
assignees: ""
---
<!-- Please describe your question here -->

View File

@@ -1,6 +0,0 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"

View File

@@ -6,82 +6,95 @@ on:
pull_request:
branches: [ master ]
permissions:
contents: read
jobs:
lint:
name: Lint
runs-on: [ubuntu-latest]
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Go 1.x
uses: actions/setup-go@v3
uses: actions/setup-go@v2
with:
go-version-file: go.work
go-version: ^1.13
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Lint
run: make lint
- name: Verify boilerplate
run: make check-license
run: ./hack/kyaml-pre-commit.sh
env:
KUSTOMIZE_DOCKER_E2E: false # don't need to do e2e tests for linting
test-linux:
name: Test Linux
runs-on: [ubuntu-latest]
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v3
- name: Set up Go 1.x
uses: actions/setup-go@v3
with:
go-version-file: go.work
id: go
- name: Test all modules
run: make test-unit-non-plugin
env:
KUSTOMIZE_DOCKER_E2E: true
- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
go-version: ^1.13
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Test kyaml
run: go test -cover ./...
working-directory: ./kyaml
- name: Test cmd/config
run: go test -cover ./...
working-directory: ./cmd/config
env:
KUSTOMIZE_DOCKER_E2E: true
test-macos:
name: Test MacOS
runs-on: [macos-latest]
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v3
- name: Set up Go 1.x
uses: actions/setup-go@v3
with:
go-version-file: go.work
id: go
- name: Test all modules
run: make test-unit-non-plugin
env:
KUSTOMIZE_DOCKER_E2E: false # docker not installed on mac
- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
go-version: ^1.13
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Test kyaml
run: go test -cover ./...
working-directory: ./kyaml
- name: Test cmd/config
run: go test -cover ./...
working-directory: ./cmd/config
env:
KUSTOMIZE_DOCKER_E2E: false # docker not installed on mac
test-windows:
name: Test Windows
runs-on: [windows-latest]
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v3
- name: Set up Go 1.x
uses: actions/setup-go@v3
with:
go-version-file: go.work
id: go
- name: Test kyaml
run: go test -cover ./...
working-directory: ./kyaml
- name: Test cmd/config
run: go test -cover ./...
working-directory: ./cmd/config
env:
KUSTOMIZE_DOCKER_E2E: false # docker on windows not working well yet
# TODO (#4001): replace specific modules above with this once Windows tests are passing.
#- name: Test all modules
# run: make test-unit-non-plugin
# env:
# KUSTOMIZE_DOCKER_E2E: false # docker on windows not working well yet
- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
go-version: ^1.13
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Test kyaml
run: go test -cover ./...
working-directory: ./kyaml
- name: Test cmd/config
run: go test -cover ./...
working-directory: ./cmd/config
env:
KUSTOMIZE_DOCKER_E2E: false # docker on windows not working well yet

10
.gitignore vendored
View File

@@ -21,13 +21,3 @@
*.DS_store
.bin
# Hugo site
publishedSite/
site/public/
site/resources/
site/.hugo_build.lock
**/node_modules/
# goreleaser artifacts
**/dist/

3
.gitmodules vendored
View File

@@ -1,3 +0,0 @@
[submodule "site/themes/docsy"]
path = site/themes/docsy
url = https://github.com/google/docsy.git

50
.golangci-kustomize.yml Normal file
View File

@@ -0,0 +1,50 @@
run:
deadline: 5m
linters:
disable-all: true
enable:
- bodyclose
- deadcode
- depguard
# - dogsled
- dupl
# - errcheck
# - funlen
# - gochecknoinits
- goconst
# - gocritic
- gocyclo
- gofmt
- goimports
- golint
- gosec
- gosimple
- govet
- ineffassign
- interfacer
- lll
- misspell
- nakedret
# - scopelint
- staticcheck
- structcheck
# stylecheck demands that acronyms not be treated as words
# in camelCase, so JsonOp become JSONOp, etc. Yuck.
# - stylecheck
- typecheck
- unconvert
- unparam
- unused
- varcheck
# - whitespace
linters-settings:
dupl:
threshold: 400
lll:
line-length: 170
gocyclo:
min-complexity: 15
golint:
min-confidence: 0.85

View File

@@ -1,83 +0,0 @@
# Copyright 2019 The Kubernetes Authors.
# SPDX-License-Identifier: Apache-2.0
run:
deadline: 5m
go: '1.20'
linters:
enable-all: true
disable:
- cyclop
- exhaustivestruct
- forbidigo
- funlen
- gci
- gocognit
- godot
- godox
- goerr113
- gofumpt
- ifshort # too many false positives
- ireturn
- nilnil
- nlreturn
- noctx
- paralleltest
- stylecheck
- varnamelen
- wsl
- exhaustruct
- deadcode
- scopelint
- nonamedreturns
- golint
- maintidx
- nosnakecase
linters-settings:
dupl:
threshold: 400
lll:
line-length: 170
gocyclo:
min-complexity: 30
revive:
rules:
- name: var-naming
arguments:
- [ "ID", "API", "JSON" ] # AllowList
- [ ] # DenyList
gomnd:
ignored-functions:
- os.WriteFile
- make
gomoddirectives:
replace-local: true
gosec:
config:
G306: "0644"
wrapcheck:
ignoreSigs:
# defaults
- .Errorf(
- errors.New(
- errors.Unwrap(
- .Wrap(
- .Wrapf(
- .WithMessage(
- .WithMessagef(
- .WithStack(
# from kyaml's errors package
- .WrapPrefixf(
issues:
new-from-rev: c94b5d8f2 # enables us to enforce a larger set of linters for new code than pass on existing code
max-same-issues: 0
exclude-rules:
- linters:
- revive
text: "don't use leading"
- linters:
- staticcheck
text: "SA1019: kioutil.Legacy"

View File

@@ -1,299 +0,0 @@
# Architecture
* _Updated: December 2021_
This document describes the repository organization and the kustomize
build process. It's meant to lower the barrier to learning and
contributing to the code base.
If not kept up to date, it will just be a historical snapshot.
## Repository layout
[human-edited docs]: https://github.com/kubernetes-sigs/cli-experimental/tree/master/site
[generated docs]: https://github.com/kubernetes-sigs/cli-experimental/tree/master/docs
[rendered docs]: https://kubectl.docs.kubernetes.io
[openapi]: https://kubernetes.io/blog/2016/12/kubernetes-supports-openapi
[`api` module]: https://github.com/kubernetes-sigs/kustomize/blob/master/api/go.mod
[`api`]: #the-api-module
[`cmd/config` module]: https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/go.mod
[`cmd/config`]: #the-cmdconfig-module
[`kustomize` module]: https://github.com/kubernetes-sigs/kustomize/blob/master/kustomize/go.mod
[`kustomize`]: #the-kustomize-module
[`kyaml` module]: https://github.com/kubernetes-sigs/kustomize/blob/master/kyaml/go.mod
[`kyaml`]: #the-kyaml-module
[`kyaml/kio.Filter`]: https://github.com/Kubernetes-sigs/kustomize/blob/master/kyaml/kio/kio.go
[`go-yaml`]: https://github.com/go-yaml/yaml/tree/v3
[3922]: https://github.com/kubernetes-sigs/kustomize/issues/3922
| directory | purpose |
| ---------: | :---------- |
| `api` | The [`api`] module, holding high level kustomize code, suitable for import by other programs. |
| `cmd` | Various Go programs aiding repo management. See also `hack`. As an outlier, includes the special [`cmd/config`] module. |
| `docs` | Old home of documentation; contains pointers to new homes: [human-edited docs], [generated docs] and [rendered docs]. |
| `examples` | Full kustomization examples that run as pre-merge tests. |
| `functions` | Examples of plugins in KRM function form. TODO([3922]): Move under `plugin`. |
| `hack` | Various shell scripts to help with code management. |
| `kustomize` | The [`kustomize`] module holds the `main.go` for kustomize. |
| `kyaml` | The [`kyaml`] module, holding Kubernetes-specific YAML editing packages used by the [`api`] module. Wraps [`go-yaml`] v3.|
| `plugin` | Examples of Kustomize plugins. |
| `releasing` | Instructions for releasing the various modules. |
| `site` | Old generated documentation, kept to provide redirection links to the new docs. |
## Modules
[semantically versioned]: https://semver.org
[Go modules]: https://github.com/golang/go/wiki/Modules
The [Go modules] in the kustomize repository are [semantically versioned].
### `kustomize`
> _Depends on [`api`], [`cmd/config`], [`kyaml`]_
The [`kustomize` module] contains the `main.go` for `kustomize`, buildable with
```
(cd kustomize; go install .)
```
[appears in kubectl]: https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/kubectl/pkg/cmd/kustomize/kustomize.go
Below this are packages containing
[cobra](http://github.com/spf13/cobra) commands implementing `build`,
`edit`, `fix`, etc., packages linked together by `main.go`.
These command packages are intentionally public, semantically
versioned, and can be used in other programs. Specifically, the
`kustomize build` command [appears in kubectl] as `kubectl kustomize`.
The code in the `build` package is dominated by flag validation,
with minimal business logic. The critical lines are something
like
```
# Make a kustomizer.
k := krusty.MakeKustomizer(
HonorKustomizeFlags(krusty.MakeDefaultOptions()),
)
# Run the kustomizer, sending location of kustomization.yaml
m := k.Run(fSys, "/path/to/dir")
# Write the result as YAML.
writer.Write(m.AsYaml())
```
The `krusty` package is in the [`api`] module.
### `api`
> _Depends on [`kyaml`] and code generated from builtin plugin modules_
The [`api` module] is used by CLI programs like `kustomize` and `kubectl`
to read and honor `kustomization.yaml` files and all that implies.
The main public packages in the [`api` module] are
| package | |
| --------: | :---------- |
| `filters` | Implementations of [`kyaml/kio.Filter`] used by kustomize to transform Kubernetes objects. |
| `konfig` | Configuration methods and constants in the kustomize API. |
| `krusty` | Primary API entry point. Holds the kustomizer and hundreds of tests for it. |
| `loader` | Loads kustomization files and the files they refer to, enforcing security rules. |
| `resmap` | The primary internal data structure over which the kustomizer and filters work. |
| `types` | The `Kustomization` object and ancillary structs. |
### `cmd/config`
> _Depends on [`kyaml`]_
This module contains cobra commands and kyaml-based functionality to
provide unix-like file manipulation commands to kustomize like `grep`
and `tree`. These commands may be included in any program that
manipulates k8s YAML (e.g. kustomize).
### `kyaml`
> _Has no in-repo dependence_
The [`kyaml` module] is a kubernetes-focussed enhancement of [go-yaml].
The YAML manipulation performed by a kustomize is based on these libraries.
These libraries evolve independently of kustomize, and other programs depend on them.
The key public packages in the [`kyaml` module] include
| package | |
| --------: | :---------- |
| `errors` | Wrapper for the go-errors/errors lib |
| `filesys` | A kustomize-specific file system abstraction, to ease writing tests |
| `fn/framework` | An SDK for writing KRM Functions in Go |
| `fn/runtime` | Implements the runtime for KRM Function extensions |
| `kio` | Libraries for reading and writing collections of Kubernetes resources as RNodes |
| `openapi` | Loads and accesses openapi schemas for schema-aware resource manipultaion |
| `resid` | Representations to aid in unique identification of Kubernetes resources |
| `yaml` | A Kubernetes-focused wrapper of [go-yaml], notably including the RNode object |
-------
## How _kustomize build_ works
The command `kustomize build` accepts a single string argument,
which must resolve to a directory, possibly in a git repository,
called the _kustomization root_.
This directory must contain a file called `kustomization.yaml`, with
YAML that marshals into a single instance of a `Kustomization` object.
For the remainder of this document, the word _kustomization_ refers to
either of these things.
This kustomization is the access point to a directed, acyclic graph of
Kubernetes objects, including other kustomizations, to include in a
build.
Execution of `build` starts and ends in the [`api`] module,
frequently dipping into the [`kyaml`] module for lower level
YAML manipulation.
### The `build` flow
- Validate command lines arguments and flags.
- Make a `Kustomizer` as a function of those arguments.
- Call `Run` on the kustomizer, passing it the path to the
kustomization.
`Run` returns an instance of `ResMap`, the `api` package's
representation of a set of kubernetes `Resource` objects.
This structure offers resource lookup methods (map behavior),
but also retains the resources in the order they were
specified in kustomization files (list behavior).
Post-run, the objects are fully hydrated, per the
instructions in the kustomization.
- Marshal the objects as YAML to a file or `stdout`.
### The `Run` function
- Create various objects
- A `ResMap` factory.
Makes `ResMaps` from byte streams, other `ResMaps`, etc.
- A file `loader.Loader`.
It's fed an appropriate set of restrictions, and the path to the kustomization.
- A plugin loader.
It finds plugins (transformers, generators or validators)
and prepares them for running.
- A `KustTarget` encapsulating all of the above.
A KustTarget contains one `Kustomization` and represents
everything that kustomization can reach. This will include
other `KustTarget` instances, each having a smaller purview than
the one referencing it.
- Call `KustTarget.Load` to load its kustomization.
This step deals with deprecations and field changes.
- Load [openapi] data specified by the kustomization.
This is needed to recognize k8s kinds and their special
properties, e.g. which kinds are cluster-scoped, which kinds
refer to others, etc.
- Call `KustTarget.makeCustomizedResmap` to create the `ResMap` result.
This visits everything referenced by the kustomization,
performing all generation, transformation and validation.
- Finish the `Run` with
- Optional reordering of objects in `ResMap`, overriding the
FIFO rule.
- Optional addition of _kustomize build annotations_ to the
resources. E.g. from which repo and file the resource was
read, the fact that kustomize touched the resource, etc.
These kustomize-specific annotations are intended for
server-side data analytics, file structure traceability and
reconstruction, etc.
### The `makeCustomizedResmap` function
This function starts the process of object transformation,
as well as accumulation of recursively referenced data.
- Call `ra := KustTarget.AccumulateTarget`.
The result, `ra`, is a resource accumulator that contains
everything referred to by the current kustomization, now fully
hydrated.
- Uniquify names of generated objects by appending content hashes.
This cannot be done until the objects are complete.
- Fix all name references (given that names may have changed).
E.g. if a ConfigMaps was given a generated name, all objects that
refer to that ConfigMap must be given its name.
- Resolve vars, replacing them with whatever they refer to (a legacy feature).
### The `AccumulateTarget` function
- Call `AccumulateResources` over the `resources` field (this can recurse).
- Call `AccumulateComponents` over the `components` field (this can recurse),
- Load legacy (pre-plugin) global kustomize configuration,
- Load legacy (pre-openapi) _Custom Resource Definition_ data.
- In the context of the data loaded above, run the kustomization's
- generators,
- transformers,
- and validators.
- Accumulate `vars` (make note of them for later replacement).
### `AccumulateResources` and component accumulation
- If the path is a file:
- Accumulate the objects in the file (treating them
as opaque kubernetes objects).
- If the path is a directory:
- Create a new `KustTarget` referring to that directory's kustomization.
- Call `subRa := KustTarget.AccumulateTarget`.
- Call `ra.MergeAccumulator(subRa)`
This completes a recursion.
- If the path is a git URL:
- Clone the repository to a temporary directory.
- Process the path optionally specified in the URL
as a path in the clone.
- If no path specified, work from the repository root.
That's as deep as this discussion will go.
The deeper this document goes into the details, the faster
it will get out of date.

View File

@@ -1,25 +1,6 @@
[SIG-CLI]: https://github.com/kubernetes/community/tree/master/sig-cli
[Slack channel]: https://kubernetes.slack.com/messages/kustomize
[Mailing list]: https://groups.google.com/forum/#!forum/kubernetes-sig-cli
[OWNERS file spec]: https://github.com/kubernetes/community/blob/master/contributors/guide/owners.md
[Kustomize OWNERS_ALIASES]: https://github.com/kubernetes-sigs/kustomize/blob/8049f7b1af52e8a7ec26faf6cf714f560d0043c5/OWNERS_ALIASES
[SIG-CLI Teams]: https://github.com/kubernetes/org/blob/main/config/kubernetes-sigs/sig-cli/teams.yaml
[Github permissions]: https://docs.github.com/en/organizations/managing-access-to-your-organizations-repositories/repository-permission-levels-for-an-organization#repository-access-for-each-permission-level
[Contributor License Agreement]: https://git.k8s.io/community/CLA.md
[Kubernetes Contributor Guide]: http://git.k8s.io/community/contributors/guide
[Contributor Cheat Sheet]: https://git.k8s.io/community/contributors/guide/contributor-cheatsheet/README.md
[CNCF Code of Conduct]: https://github.com/cncf/foundation/blob/master/code-of-conduct.md
[Kubernetes Community Membership]: https://github.com/kubernetes/community/blob/master/community-membership.md
[Contribution Guide]: https://kubectl.docs.kubernetes.io/contributing/kustomize/
[MacOS Dev Guide]: https://kubectl.docs.kubernetes.io/contributing/kustomize/mac/
[Windows Dev Guide]: https://kubectl.docs.kubernetes.io/contributing/kustomize/windows/
# Contributing Guidelines
Welcome to Kubernetes. We are excited about the prospect of you joining our [community](https://github.com/kubernetes/community)! The Kubernetes community abides by the [CNCF Code of Conduct]. Here is an excerpt:
Welcome to Kubernetes. We are excited about the prospect of you joining our [community](https://github.com/kubernetes/community)! The Kubernetes community abides by the CNCF [code of conduct](code-of-conduct.md). Here is an excerpt:
_As contributors and maintainers of this project, and in the interest of fostering an open and welcoming community, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities._
@@ -27,22 +8,13 @@ _As contributors and maintainers of this project, and in the interest of fosteri
Dev guides:
- [Contribution Guide]
- [MacOS Dev Guide]
- [Windows Dev Guide]
- [Mac](docs/macDevGuide.md)
General resources for contributors:
We have full documentation on how to get started contributing here:
- [Contributor License Agreement] - Kubernetes projects require that you sign a Contributor License Agreement (CLA) before we can accept your pull requests.
- [Kubernetes Contributor Guide] - Main contributor documentation.
- [Contributor Cheat Sheet] - Common resources for existing developers.
Here are some additional ideas to help you get started with Kustomize:
- Attend a Kustomize Bug Scrub. Check the [SIG-CLI] meetings list to find the next one.
- Help triage issues by confirming validity and applying the appropriate `kind` label (e.g. comment `/kind bug`).
- Pick up an issue to fix. Issues with the `help-wanted` label are a good place to start, but you can also look for any issue with the `triage/accepted` label and no assignee. Remember to `/assign` yourself to let others know you're working on it.
- Help confirm new issues labelled `kind/bug` by reproducing them with the latest release.
- Support Kustomize users by responding to questions on issues labelled `kind/support` or in the [Slack channel].
- [Contributor License Agreement](https://git.k8s.io/community/CLA.md) Kubernetes projects require that you sign a Contributor License Agreement (CLA) before we can accept your pull requests
- [Kubernetes Contributor Guide](http://git.k8s.io/community/contributors/guide) - Main contributor documentation, or you can just jump directly to the [contributing section](http://git.k8s.io/community/contributors/guide#contributing)
- [Contributor Cheat Sheet](https://git.k8s.io/community/contributors/guide/contributor-cheatsheet/README.md) - Common resources for existing developers
## Mentorship
@@ -50,22 +22,19 @@ Here are some additional ideas to help you get started with Kustomize:
## Contributor Ladder
Kustomize follows the [Kubernetes Community Membership] contributor ladder. Roles are as follows:
Kustomize generally follows the [Kubernetes Community Membership](https://github.com/kubernetes/community/blob/master/community-membership.md) contributor ladder. Roles are as follows:
1. Contributor: Anyone who actively contributes code, issues or reviews to the project. All contributors must sign the [Contributor License Agreement].
1. Reviewer: Contributors with a history of review and authorship on Kustomize. Has LGTM rights on the Kustomize repo (as do all kubernetes-sigs org members). Active contributors are encouraged to join the reviewers list to be automatically pinged on PRs.
1. Approver: Highly experienced active reviewer and contributor to Kustomize. Has both LTGM and approval rights on the Kustomize repo, as well as "maintain" [Github permissions].
1. Owner: Approver who sets technical direction and makes or approves design decisions for the project. Has LGTM and approval rights on the Kustomize repo as well as "admin" [Github permissions].
The kyaml module within the Kustomize repo has additional owners following the same ladder.
1. Contributor: Anyone who actively contributes code, issues or reviews to the project. There are no Kustomize-specific requirements for this status. All contributors must [sign the CLA](https://github.com/kubernetes/community/tree/master/contributors/guide#prerequisites).
1. Member/Reviewer: All Kubernetes-SIGs org members have LGTM rights on the Kustomize repo. There are no Kustomize-specific requirements. Kustomize does not currently have any formal reviewers, but the role will be created if there is interest.
1. Maintainer/Approver: Highly experienced active reviewer and contributor to Kustomize. Has both LTGM and approval rights on the Kustomize repo, as well as [Github "maintain" rights](https://docs.github.com/en/organizations/managing-access-to-your-organizations-repositories/repository-permission-levels-for-an-organization#repository-access-for-each-permission-level).
1. Admin/Owner: Maintainer who sets technical direction and makes or approves design decisions for the project. Has LGTM and approval rights on the Kustomize repo as well as [Github "admin" rights](https://docs.github.com/en/organizations/managing-access-to-your-organizations-repositories/repository-permission-levels-for-an-organization#repository-access-for-each-permission-level).
Administrative notes:
- The [OWNERS file spec] is a useful resources in making changes.
- Maintainers and admins must be added to the appropriate lists in both [Kustomize OWNERS_ALIASES] and [SIG-CLI Teams]. If this isn't done, the individual in question will lack either PR approval rights (Kustomize list) or the appropriate Github repository permissions (community list).
- Maintainers and admins must be added to the appropriate list both [in the Kustomize repo](https://github.com/kubernetes-sigs/kustomize/blob/8049f7b1af52e8a7ec26faf6cf714f560d0043c5/OWNERS_ALIASES) and [in the community repo](https://github.com/kubernetes/org/blob/main/config/kubernetes-sigs/sig-cli/teams.yaml). If this isn't done, the individual in question will lack either PR approval rights (Kustomize list) or the appropriate Github repository permissions (community list).
- The spec for the OWNERS file is [in the community repo](https://github.com/kubernetes/community/blob/master/contributors/guide/owners.md).
## Contact Information
- [Slack channel]
- [Mailing list]
- [Slack channel](https://kubernetes.slack.com/messages/sig-cli)
- [Mailing list](https://groups.google.com/forum/#!forum/kubernetes-sig-cli)

373
Makefile
View File

@@ -3,16 +3,13 @@
#
# Makefile for kustomize CLI and API.
LATEST_RELEASE=v5.0.3
SHELL := /usr/bin/env bash
GOOS = $(shell go env GOOS)
GOARCH = $(shell go env GOARCH)
MYGOBIN = $(shell go env GOBIN)
ifeq ($(MYGOBIN),)
MYGOBIN = $(shell go env GOPATH)/bin
endif
export PATH := $(MYGOBIN):$(PATH)
MODULES := '"cmd/config" "api/" "kustomize/" "kyaml/"'
# Provide defaults for REPO_OWNER and REPO_NAME if not present.
# Typically these values would be provided by Prow.
@@ -24,35 +21,49 @@ ifndef REPO_NAME
REPO_NAME := "kustomize"
endif
.PHONY: all
all: install-tools verify-kustomize
# --- Plugins ---
include Makefile-plugins.mk
.PHONY: verify-kustomize
verify-kustomize: \
lint-kustomize \
test-unit-kustomize-all \
test-examples-kustomize-against-HEAD \
test-examples-kustomize-against-4.1
# The following target referenced by a file in
# https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes-sigs/kustomize
.PHONY: prow-presubmit-check
prow-presubmit-check: \
install-tools \
lint-kustomize \
test-multi-module \
test-unit-kustomize-all \
test-unit-cmd-all \
test-go-mod \
test-examples-kustomize-against-HEAD \
test-examples-kustomize-against-4.1
# --- Tool management ---
include Makefile-tools.mk
.PHONY: verify-kustomize-e2e
verify-kustomize-e2e: test-examples-e2e-kustomize
.PHONY: install-tools
install-tools: \
install-local-tools \
install-out-of-tree-tools
# Other builds in this repo might want a different linter version.
# Without one Makefile to rule them all, the different makes
# cannot assume that golanci-lint is at the version they want.
# This installs what kustomize wants to use.
$(MYGOBIN)/golangci-lint-kustomize:
rm -f $(CURDIR)/hack/golangci-lint
GOBIN=$(CURDIR)/hack go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.23.8
mv $(CURDIR)/hack/golangci-lint $(MYGOBIN)/golangci-lint-kustomize
.PHONY: uninstall-tools
uninstall-tools: \
uninstall-local-tools \
uninstall-out-of-tree-tools
$(MYGOBIN)/mdrip:
go install github.com/monopole/mdrip@v1.0.2
.PHONY: install-local-tools
install-local-tools: \
$(MYGOBIN)/gorepomod \
$(MYGOBIN)/k8scopy \
$(MYGOBIN)/pluginator
$(MYGOBIN)/stringer:
go get golang.org/x/tools/cmd/stringer
.PHONY: uninstall-local-tools
uninstall-local-tools:
rm -f $(MYGOBIN)/gorepomod
rm -f $(MYGOBIN)/k8scopy
rm -f $(MYGOBIN)/pluginator
$(MYGOBIN)/goimports:
go get golang.org/x/tools/cmd/goimports
# Build from local source.
$(MYGOBIN)/gorepomod:
@@ -69,94 +80,189 @@ $(MYGOBIN)/pluginator:
cd cmd/pluginator; \
go install .
# --- Build targets ---
# Build from local source.
$(MYGOBIN)/prchecker:
cd cmd/prchecker; \
go install .
# Build from local source.
$(MYGOBIN)/kustomize: build-kustomize-api
cd kustomize; \
go install -ldflags "-X sigs.k8s.io/kustomize/api/provenance.buildDate=$(shell date -u +'%Y-%m-%dT%H:%M:%SZ')" \
.
go install .
kustomize: $(MYGOBIN)/kustomize
.PHONY: install-tools
install-tools: \
$(MYGOBIN)/goimports \
$(MYGOBIN)/golangci-lint-kustomize \
$(MYGOBIN)/gorepomod \
$(MYGOBIN)/helmV3 \
$(MYGOBIN)/k8scopy \
$(MYGOBIN)/mdrip \
$(MYGOBIN)/pluginator \
$(MYGOBIN)/prchecker \
$(MYGOBIN)/stringer
### Begin kustomize plugin rules.
#
# The rules to deal with builtin plugins are a bit
# complicated because
#
# - Every builtin plugin is a Go plugin -
# meaning it gets its own module directory
# (outside of the api module) with Go
# code in a 'main' package per Go plugin rules.
# - kustomize locates plugins using the
# 'apiVersion' and 'kind' fields from the
# plugin config file.
# - k8s wants CamelCase in 'kind' fields.
# - The module name (the last name in the path)
# must be the lowercased 'kind' of the
# plugin because Go and related tools
# demand lowercase in import paths, but
# allow CamelCase in file names.
# - the generated code must live in the api
# module (it's linked into the api).
# Where all generated builtin plugin code should go.
pGen=api/builtins
# Where the builtin Go plugin modules live.
pSrc=plugin/builtin
_builtinplugins = \
AnnotationsTransformer.go \
ConfigMapGenerator.go \
IAMPolicyGenerator.go \
HashTransformer.go \
ImageTagTransformer.go \
LabelTransformer.go \
LegacyOrderTransformer.go \
NamespaceTransformer.go \
PatchJson6902Transformer.go \
PatchStrategicMergeTransformer.go \
PatchTransformer.go \
PrefixSuffixTransformer.go \
ReplacementTransformer.go \
ReplicaCountTransformer.go \
SecretGenerator.go \
ValueAddTransformer.go \
HelmChartInflationGenerator.go
# Maintaining this explicit list of generated files, and
# adding it as a dependency to a few targets, to assure
# they get recreated if deleted. The rules below on how
# to make them don't, by themselves, assure they will be
# recreated if deleted.
builtinplugins = $(patsubst %,$(pGen)/%,$(_builtinplugins))
# These rules are verbose, but assure that if a source file
# is modified, the corresponding generated file, and only
# that file, will be recreated.
$(pGen)/AnnotationsTransformer.go: $(pSrc)/annotationstransformer/AnnotationsTransformer.go
$(pGen)/ConfigMapGenerator.go: $(pSrc)/configmapgenerator/ConfigMapGenerator.go
$(pGen)/GkeSaGenerator.go: $(pSrc)/gkesagenerator/GkeSaGenerator.go
$(pGen)/HashTransformer.go: $(pSrc)/hashtransformer/HashTransformer.go
$(pGen)/ImageTagTransformer.go: $(pSrc)/imagetagtransformer/ImageTagTransformer.go
$(pGen)/LabelTransformer.go: $(pSrc)/labeltransformer/LabelTransformer.go
$(pGen)/LegacyOrderTransformer.go: $(pSrc)/legacyordertransformer/LegacyOrderTransformer.go
$(pGen)/NamespaceTransformer.go: $(pSrc)/namespacetransformer/NamespaceTransformer.go
$(pGen)/PatchJson6902Transformer.go: $(pSrc)/patchjson6902transformer/PatchJson6902Transformer.go
$(pGen)/PatchStrategicMergeTransformer.go: $(pSrc)/patchstrategicmergetransformer/PatchStrategicMergeTransformer.go
$(pGen)/PatchTransformer.go: $(pSrc)/patchtransformer/PatchTransformer.go
$(pGen)/PrefixSuffixTransformer.go: $(pSrc)/prefixsuffixtransformer/PrefixSuffixTransformer.go
$(pGen)/ReplacementTransformer.go: $(pSrc)/replacementtransformer/ReplacementTransformer.go
$(pGen)/ReplicaCountTransformer.go: $(pSrc)/replicacounttransformer/ReplicaCountTransformer.go
$(pGen)/SecretGenerator.go: $(pSrc)/secretgenerator/SecretGenerator.go
$(pGen)/ValueAddTransformer.go: $(pSrc)/valueaddtransformer/ValueAddTransformer.go
$(pGen)/HelmChartInflationGenerator.go: $(pSrc)/helmchartinflationgenerator/HelmChartInflationGenerator.go
# The (verbose but portable) Makefile way to convert to lowercase.
toLowerCase = $(subst A,a,$(subst B,b,$(subst C,c,$(subst D,d,$(subst E,e,$(subst F,f,$(subst G,g,$(subst H,h,$(subst I,i,$(subst J,j,$(subst K,k,$(subst L,l,$(subst M,m,$(subst N,n,$(subst O,o,$(subst P,p,$(subst Q,q,$(subst R,r,$(subst S,s,$(subst T,t,$(subst U,u,$(subst V,v,$(subst W,w,$(subst X,x,$(subst Y,y,$(subst Z,z,$1))))))))))))))))))))))))))
$(pGen)/%.go: $(MYGOBIN)/pluginator
@echo "generating $*"
( \
set -e; \
cd $(pSrc)/$(call toLowerCase,$*); \
go generate .; \
cd ../../../$(pGen); \
$(MYGOBIN)/goimports -w $*.go \
)
# Target is for debugging.
.PHONY: generate-kustomize-builtin-plugins
generate-kustomize-builtin-plugins: $(builtinplugins)
.PHONY: build-kustomize-external-go-plugin
build-kustomize-external-go-plugin:
./hack/buildExternalGoPlugins.sh ./plugin
.PHONY: clean-kustomize-external-go-plugin
clean-kustomize-external-go-plugin:
./hack/buildExternalGoPlugins.sh ./plugin clean
### End kustomize plugin rules.
.PHONY: lint-kustomize
lint-kustomize: $(MYGOBIN)/golangci-lint-kustomize $(builtinplugins)
cd api; $(MYGOBIN)/golangci-lint-kustomize \
-c ../.golangci-kustomize.yml \
run ./...
cd kustomize; $(MYGOBIN)/golangci-lint-kustomize \
-c ../.golangci-kustomize.yml \
run ./...
cd cmd/pluginator; $(MYGOBIN)/golangci-lint-kustomize \
-c ../../.golangci-kustomize.yml \
run ./...
# Used to add non-default compilation flags when experimenting with
# plugin-to-api compatibility checks.
.PHONY: build-kustomize-api
build-kustomize-api: $(MYGOBIN)/goimports $(builtinplugins)
cd api; $(MAKE) build
build-kustomize-api: $(builtinplugins)
cd api; go build ./...
.PHONY: generate-kustomize-api
generate-kustomize-api:
cd api; $(MAKE) generate
generate-kustomize-api: $(MYGOBIN)/k8scopy
cd api; go generate ./...
# --- Verification targets ---
.PHONY: verify-kustomize-repo
verify-kustomize-repo: \
install-tools \
lint \
check-license \
test-unit-all \
build-non-plugin-all \
test-go-mod \
test-examples-kustomize-against-HEAD \
test-examples-kustomize-against-latest-release
# The following target referenced by a file in
# https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes-sigs/kustomize
.PHONY: prow-presubmit-check
prow-presubmit-check: \
install-tools \
test-unit-kustomize-plugins \
test-go-mod \
build-non-plugin-all \
test-examples-kustomize-against-HEAD \
test-examples-kustomize-against-latest-release
.PHONY: license
license: $(MYGOBIN)/addlicense
./hack/add-license.sh run
.PHONY: check-license
check-license: $(MYGOBIN)/addlicense
./hack/add-license.sh check
.PHONY: lint
lint: $(MYGOBIN)/golangci-lint $(MYGOBIN)/goimports $(builtinplugins)
./hack/for-each-module.sh "make lint"
.PHONY: test-unit-all
test-unit-all: \
test-unit-non-plugin \
test-unit-kustomize-plugins
# This target is used by our Github Actions CI to run unit tests for all non-plugin modules in multiple GOOS environments.
.PHONY: test-unit-non-plugin
test-unit-non-plugin:
./hack/for-each-module.sh "make test" "./plugin/*" 16
.PHONY: build-non-plugin-all
build-non-plugin-all:
./hack/for-each-module.sh "make build" "./plugin/*" 16
.PHONY: test-unit-kustomize-api
test-unit-kustomize-api: build-kustomize-api
cd api; go test ./... -ldflags "-X sigs.k8s.io/kustomize/api/provenance.version=v444.333.222"
.PHONY: test-unit-kustomize-plugins
test-unit-kustomize-plugins:
./hack/testUnitKustomizePlugins.sh
.PHONY: functions-examples-all
functions-examples-all:
for dir in $(abspath $(wildcard functions/examples/*/.)); do \
echo -e "\n---Running make tasks for function $$dir---"; \
set -e; \
cd $$dir; $(MAKE) all; \
done
.PHONY: test-unit-kustomize-cli
test-unit-kustomize-cli:
cd kustomize; go test ./...
.PHONY: test-unit-kustomize-all
test-unit-kustomize-all: \
test-unit-kustomize-api \
test-unit-kustomize-cli \
test-unit-kustomize-plugins
test-unit-cmd-all:
./hack/kyaml-pre-commit.sh
test-go-mod:
./hack/for-each-module.sh "go mod tidy -v"
./hack/check-go-mod.sh
# Environment variables are defined at
# https://github.com/kubernetes/test-infra/blob/master/prow/jobs.md#job-environment-variables
.PHONY: test-multi-module
test-multi-module: $(MYGOBIN)/prchecker
( \
export MYGOBIN=$(MYGOBIN); \
export REPO_OWNER=$(REPO_OWNER); \
export REPO_NAME=$(REPO_NAME); \
export PULL_NUMBER=$(PULL_NUMBER); \
export MODULES=$(MODULES); \
./hack/check-multi-module.sh; \
)
.PHONY:
verify-kustomize-e2e: $(MYGOBIN)/mdrip $(MYGOBIN)/kind
test-examples-e2e-kustomize: $(MYGOBIN)/mdrip $(MYGOBIN)/kind
( \
set -e; \
/bin/rm -f $(MYGOBIN)/kustomize; \
@@ -170,16 +276,89 @@ test-examples-kustomize-against-HEAD: $(MYGOBIN)/kustomize $(MYGOBIN)/mdrip
./hack/testExamplesAgainstKustomize.sh HEAD
.PHONY:
test-examples-kustomize-against-latest-release: $(MYGOBIN)/mdrip
./hack/testExamplesAgainstKustomize.sh v5@$(LATEST_RELEASE)
test-examples-kustomize-against-4.1: $(MYGOBIN)/mdrip
./hack/testExamplesAgainstKustomize.sh v4@v4.1.2
# linux only.
# This is for testing an example plugin that
# uses kubeval for validation.
# Don't want to add a hard dependence in go.mod file
# to github.com/instrumenta/kubeval.
# Instead, download the binary.
$(MYGOBIN)/kubeval:
( \
set -e; \
d=$(shell mktemp -d); cd $$d; \
wget https://github.com/instrumenta/kubeval/releases/latest/download/kubeval-linux-amd64.tar.gz; \
tar xf kubeval-linux-amd64.tar.gz; \
mv kubeval $(MYGOBIN); \
rm -rf $$d; \
)
# linux only.
# This is for testing an example plugin that uses helm to inflate a chart
# for subsequent kustomization.
# Don't want to add a hard dependence in go.mod file to helm.
# Instead, download the binaries.
$(MYGOBIN)/helmV2:
( \
set -e; \
d=$(shell mktemp -d); cd $$d; \
tgzFile=helm-v2.13.1-linux-amd64.tar.gz; \
wget https://storage.googleapis.com/kubernetes-helm/$$tgzFile; \
tar -xvzf $$tgzFile; \
mv linux-amd64/helm $(MYGOBIN)/helmV2; \
rm -rf $$d \
)
# Helm V3 differs from helm V2; downloading it to provide coverage for the
# chart inflator plugin under helm v3.
$(MYGOBIN)/helmV3:
( \
set -e; \
d=$(shell mktemp -d); cd $$d; \
tgzFile=helm-v3.5.3-linux-amd64.tar.gz; \
wget https://get.helm.sh/$$tgzFile; \
tar -xvzf $$tgzFile; \
mv linux-amd64/helm $(MYGOBIN)/helmV3; \
rm -rf $$d \
)
$(MYGOBIN)/kind:
( \
set -e; \
d=$(shell mktemp -d); cd $$d; \
wget -O ./kind https://github.com/kubernetes-sigs/kind/releases/download/v0.7.0/kind-$(shell uname)-amd64; \
chmod +x ./kind; \
mv ./kind $(MYGOBIN); \
rm -rf $$d; \
)
# linux only.
$(MYGOBIN)/gh:
( \
set -e; \
d=$(shell mktemp -d); cd $$d; \
tgzFile=gh_1.0.0_linux_amd64.tar.gz; \
wget https://github.com/cli/cli/releases/download/v1.0.0/$$tgzFile; \
tar -xvzf $$tgzFile; \
mv gh_1.0.0_linux_amd64/bin/gh $(MYGOBIN)/gh; \
rm -rf $$d \
)
# --- Cleanup targets ---
.PHONY: clean
clean: clean-kustomize-external-go-plugin uninstall-tools
clean: clean-kustomize-external-go-plugin
go clean --cache
rm -f $(builtinplugins)
rm -f $(MYGOBIN)/goimports
rm -f $(MYGOBIN)/golangci-lint-kustomize
rm -f $(MYGOBIN)/kustomize
rm -f $(MYGOBIN)/mdrip
rm -f $(MYGOBIN)/prchecker
rm -f $(MYGOBIN)/stringer
# Handle pluginator manually.
# rm -f $(MYGOBIN)/pluginator
# Nuke the site from orbit. It's the only way to be sure.
.PHONY: nuke

View File

@@ -1,39 +0,0 @@
# Copyright 2022 The Kubernetes Authors.
# SPDX-License-Identifier: Apache-2.0
MYGOBIN = $(shell go env GOBIN)
ifeq ($(MYGOBIN),)
MYGOBIN = $(shell go env GOPATH)/bin
endif
export PATH := $(MYGOBIN):$(PATH)
# only set this if not already set, so importing makefiles can override it
export KUSTOMIZE_ROOT ?= $(shell pwd | sed -E 's|(.*\/kustomize)/(.*)|\1|')
include $(KUSTOMIZE_ROOT)/Makefile-tools.mk
.PHONY: lint test fix fmt tidy vet build
lint: $(MYGOBIN)/golangci-lint
$(MYGOBIN)/golangci-lint cache clean # Workaround for https://github.com/golangci/golangci-lint/issues/3228
$(MYGOBIN)/golangci-lint \
-c $$KUSTOMIZE_ROOT/.golangci.yml \
--path-prefix $(shell pwd | sed -E 's|(.*\/kustomize)/(.*)|\2|') \
run ./...
test:
go test -v -timeout 45m -cover ./...
fix:
go fix ./...
fmt:
go fmt ./...
tidy:
go mod tidy
vet:
go vet ./...
build:
go build -v -o $(MYGOBIN) ./...

View File

@@ -1,102 +0,0 @@
# Copyright 2022 The Kubernetes Authors.
# SPDX-License-Identifier: Apache-2.0
### Kustomize plugin rules.
#
# The rules to deal with builtin plugins are a bit
# complicated because
#
# - Every builtin plugin is a Go plugin -
# meaning it gets its own module directory
# (outside of the api module) with Go
# code in a 'main' package per Go plugin rules.
# - kustomize locates plugins using the
# 'apiVersion' and 'kind' fields from the
# plugin config file.
# - k8s wants CamelCase in 'kind' fields.
# - The module name (the last name in the path)
# must be the lowercased 'kind' of the
# plugin because Go and related tools
# demand lowercase in import paths, but
# allow CamelCase in file names.
# - the generated code must live in the api
# module (it's linked into the api).
# Where all generated builtin plugin code should go.
pGen=api/internal/builtins
# Where the builtin Go plugin modules live.
pSrc=plugin/builtin
_builtinplugins = \
AnnotationsTransformer.go \
ConfigMapGenerator.go \
IAMPolicyGenerator.go \
HashTransformer.go \
ImageTagTransformer.go \
LabelTransformer.go \
SortOrderTransformer.go \
NamespaceTransformer.go \
PatchJson6902Transformer.go \
PatchStrategicMergeTransformer.go \
PatchTransformer.go \
PrefixTransformer.go \
SuffixTransformer.go \
ReplacementTransformer.go \
ReplicaCountTransformer.go \
SecretGenerator.go \
ValueAddTransformer.go \
HelmChartInflationGenerator.go
# Maintaining this explicit list of generated files, and
# adding it as a dependency to a few targets, to assure
# they get recreated if deleted. The rules below on how
# to make them don't, by themselves, assure they will be
# recreated if deleted.
builtinplugins = $(patsubst %,$(pGen)/%,$(_builtinplugins))
# These rules are verbose, but assure that if a source file
# is modified, the corresponding generated file, and only
# that file, will be recreated.
$(pGen)/AnnotationsTransformer.go: $(pSrc)/annotationstransformer/AnnotationsTransformer.go
$(pGen)/ConfigMapGenerator.go: $(pSrc)/configmapgenerator/ConfigMapGenerator.go
$(pGen)/GkeSaGenerator.go: $(pSrc)/gkesagenerator/GkeSaGenerator.go
$(pGen)/HashTransformer.go: $(pSrc)/hashtransformer/HashTransformer.go
$(pGen)/ImageTagTransformer.go: $(pSrc)/imagetagtransformer/ImageTagTransformer.go
$(pGen)/LabelTransformer.go: $(pSrc)/labeltransformer/LabelTransformer.go
$(pGen)/SortOrderTransformer.go: $(pSrc)/sortordertransformer/SortOrderTransformer.go
$(pGen)/NamespaceTransformer.go: $(pSrc)/namespacetransformer/NamespaceTransformer.go
$(pGen)/PatchJson6902Transformer.go: $(pSrc)/patchjson6902transformer/PatchJson6902Transformer.go
$(pGen)/PatchStrategicMergeTransformer.go: $(pSrc)/patchstrategicmergetransformer/PatchStrategicMergeTransformer.go
$(pGen)/PatchTransformer.go: $(pSrc)/patchtransformer/PatchTransformer.go
$(pGen)/PrefixTransformer.go: $(pSrc)/prefixtransformer/PrefixTransformer.go
$(pGen)/SuffixTransformer.go: $(pSrc)/suffixtransformer/SuffixTransformer.go
$(pGen)/ReplacementTransformer.go: $(pSrc)/replacementtransformer/ReplacementTransformer.go
$(pGen)/ReplicaCountTransformer.go: $(pSrc)/replicacounttransformer/ReplicaCountTransformer.go
$(pGen)/SecretGenerator.go: $(pSrc)/secretgenerator/SecretGenerator.go
$(pGen)/ValueAddTransformer.go: $(pSrc)/valueaddtransformer/ValueAddTransformer.go
$(pGen)/HelmChartInflationGenerator.go: $(pSrc)/helmchartinflationgenerator/HelmChartInflationGenerator.go
# The (verbose but portable) Makefile way to convert to lowercase.
toLowerCase = $(subst A,a,$(subst B,b,$(subst C,c,$(subst D,d,$(subst E,e,$(subst F,f,$(subst G,g,$(subst H,h,$(subst I,i,$(subst J,j,$(subst K,k,$(subst L,l,$(subst M,m,$(subst N,n,$(subst O,o,$(subst P,p,$(subst Q,q,$(subst R,r,$(subst S,s,$(subst T,t,$(subst U,u,$(subst V,v,$(subst W,w,$(subst X,x,$(subst Y,y,$(subst Z,z,$1))))))))))))))))))))))))))
$(pGen)/%.go: $(MYGOBIN)/pluginator $(MYGOBIN)/goimports
@echo "generating $*"
( \
set -e; \
cd $(pSrc)/$(call toLowerCase,$*); \
go generate .; \
cd ../../../$(pGen); \
$(MYGOBIN)/goimports -w $*.go \
)
# Target is for debugging.
.PHONY: generate-kustomize-builtin-plugins
generate-kustomize-builtin-plugins: $(builtinplugins)
.PHONY: build-kustomize-external-go-plugin
build-kustomize-external-go-plugin:
./hack/buildExternalGoPlugins.sh ./plugin
.PHONY: clean-kustomize-external-go-plugin
clean-kustomize-external-go-plugin:
./hack/buildExternalGoPlugins.sh ./plugin clean

View File

@@ -1,102 +0,0 @@
# Copyright 2022 The Kubernetes Authors.
# SPDX-License-Identifier: Apache-2.0
GOLANGCI_LINT_VERSION=v1.51.2
MYGOBIN = $(shell go env GOBIN)
ifeq ($(MYGOBIN),)
MYGOBIN = $(shell go env GOPATH)/bin
endif
export PATH := $(MYGOBIN):$(PATH)
# determines whether to run tests that only behave locally; can be overridden by override variable
export IS_LOCAL = false
.PHONY: install-out-of-tree-tools
install-out-of-tree-tools: \
$(MYGOBIN)/goimports \
$(MYGOBIN)/golangci-lint \
$(MYGOBIN)/helmV3 \
$(MYGOBIN)/mdrip \
$(MYGOBIN)/stringer \
$(MYGOBIN)/goimports
.PHONY: uninstall-out-of-tree-tools
uninstall-out-of-tree-tools:
rm -f $(MYGOBIN)/goimports
rm -f $(MYGOBIN)/golangci-lint
rm -f $(MYGOBIN)/helmV3
rm -f $(MYGOBIN)/mdrip
rm -f $(MYGOBIN)/stringer
$(MYGOBIN)/golangci-lint:
go install github.com/golangci/golangci-lint/cmd/golangci-lint@$(GOLANGCI_LINT_VERSION)
$(MYGOBIN)/mdrip:
go install github.com/monopole/mdrip@v1.0.2
$(MYGOBIN)/stringer:
go install golang.org/x/tools/cmd/stringer@latest
$(MYGOBIN)/goimports:
go install golang.org/x/tools/cmd/goimports@latest
$(MYGOBIN)/mdtogo:
go install sigs.k8s.io/kustomize/cmd/mdtogo@latest
$(MYGOBIN)/addlicense:
go install github.com/google/addlicense@latest
$(MYGOBIN)/goreleaser:
go install github.com/goreleaser/goreleaser@v0.179.0 # https://github.com/kubernetes-sigs/kustomize/issues/4542
$(MYGOBIN)/kind:
( \
set -e; \
d=$(shell mktemp -d); cd $$d; \
wget -O ./kind https://github.com/kubernetes-sigs/kind/releases/download/v0.7.0/kind-$(GOOS)-$(GOARCH); \
chmod +x ./kind; \
mv ./kind $(MYGOBIN); \
rm -rf $$d; \
)
# linux only.
$(MYGOBIN)/gh:
( \
set -e; \
d=$(shell mktemp -d); cd $$d; \
tgzFile=gh_1.0.0_$(GOOS)_$(GOARCH).tar.gz; \
wget https://github.com/cli/cli/releases/download/v1.0.0/$$tgzFile; \
tar -xvzf $$tgzFile; \
mv gh_1.0.0_$(GOOS)_$(GOARCH)/bin/gh $(MYGOBIN)/gh; \
rm -rf $$d \
)
# linux only.
# This is for testing an example plugin that
# uses kubeval for validation.
# Don't want to add a hard dependence in go.mod file
# to github.com/instrumenta/kubeval.
# Instead, download the binary.
$(MYGOBIN)/kubeval:
( \
set -e; \
d=$(shell mktemp -d); cd $$d; \
wget https://github.com/instrumenta/kubeval/releases/latest/download/kubeval-$(GOOS)-$(GOARCH).tar.gz; \
tar xf kubeval-$(GOOS)-$(GOARCH).tar.gz; \
mv kubeval $(MYGOBIN); \
rm -rf $$d; \
)
# Helm V3 differs from helm V2; downloading it to provide coverage for the
# chart inflator plugin under helm v3.
$(MYGOBIN)/helmV3:
( \
set -e; \
d=$(shell mktemp -d); cd $$d; \
tgzFile=helm-v3.10.2-$(GOOS)-$(GOARCH).tar.gz; \
wget https://get.helm.sh/$$tgzFile; \
tar -xvzf $$tgzFile; \
mv $(GOOS)-$(GOARCH)/helm $(MYGOBIN)/helmV3; \
rm -rf $$d \
)

5
OWNERS
View File

@@ -1,5 +1,4 @@
# See https://github.com/kubernetes/community/blob/master/community-membership.md
approvers:
- kustomize-approvers
reviewers:
- kustomize-reviewers
- kustomize-admins
- kustomize-maintainers

View File

@@ -1,25 +1,14 @@
# Keep *-owners and *-approvers lists in sync with *-admins and *-maintainers in
# https://github.com/kubernetes/org/blob/main/config/kubernetes-sigs/sig-cli/teams.yaml
aliases:
kustomize-owners:
kustomize-admins: # Please keep in sync with kustomize-admins in https://github.com/kubernetes/org/blob/main/config/kubernetes-sigs/sig-cli/teams.yaml
- knverey
- monopole
- pwittrock
kustomize-maintainers: # Please keep in sync with kustomize-maintainers in https://github.com/kubernetes/org/blob/main/config/kubernetes-sigs/sig-cli/teams.yaml
- justinsb
- mortent
- natasha41575
kustomize-approvers:
- knverey
- natasha41575
- annasong20
kustomize-reviewers:
- knverey
- natasha41575
- yuwenma
- annasong20
- koba1t
# emeritus:
# - liujingfang1
# - Shell32-Natsu
# - justinsb
# - monopole
# - pwittrock
# - mengqiy
# - mortent
# - phanimarupaka
- phanimarupaka
- Shell32-Natsu
emeritus-maintainers:
- liujingfang1
- mengqiy

View File

@@ -11,8 +11,8 @@ and it's like [`sed`], in that it emits edited text.
This tool is sponsored by [sig-cli] ([KEP]).
- [Installation instructions](https://kubectl.docs.kubernetes.io/installation/kustomize/)
- [General documentation](https://kubectl.docs.kubernetes.io/references/kustomize/)
- [Installation instructions](https://kubernetes-sigs.github.io/kustomize/installation)
- [General documentation](https://kubernetes-sigs.github.io/kustomize)
- [Examples](examples)
[![Build Status](https://prow.k8s.io/badge.svg?jobs=kustomize-presubmit-master)](https://prow.k8s.io/job-history/kubernetes-jenkins/pr-logs/directory/kustomize-presubmit-master)
@@ -20,35 +20,20 @@ This tool is sponsored by [sig-cli] ([KEP]).
## kubectl integration
To find the kustomize version embedded in recent versions of kubectl, run `kubectl version`:
```sh
> kubectl version --short --client
Client Version: v1.26.0
Kustomize Version: v4.5.7
```
The kustomize build flow at [v2.0.3] was added
to [kubectl v1.14][kubectl announcement]. The kustomize
flow in kubectl remained frozen at v2.0.3 until kubectl v1.21,
which [updated it to v4.0.5][kust-in-kubectl update]. It will
be updated on a regular basis going forward, and such updates
will be reflected in the Kubernetes release notes.
flow in kubectl has remained frozen at v2.0.3 while work
to extract kubectl from the k/k repo, and work to remove
kustomize's dependence on core k/k code ([#2506]) has proceeded.
The reintegration effort is tracked in [#1500] (and its blocking
issues).
| Kubectl version | Kustomize version |
| --- | --- |
| < v1.14 | n/a |
| v1.14-v1.20 | v2.0.3 |
| v1.21 | v4.0.5 |
| v1.22 | v4.2.0 |
[v2.0.3]: https://github.com/kubernetes-sigs/kustomize/releases/tag/v2.0.3
[v2.0.3]: /../../tree/v2.0.3
[#2506]: https://github.com/kubernetes-sigs/kustomize/issues/2506
[#1500]: https://github.com/kubernetes-sigs/kustomize/issues/1500
[kust-in-kubectl update]: https://github.com/kubernetes/kubernetes/blob/4d75a6238a6e330337526e0513e67d02b1940b63/CHANGELOG/CHANGELOG-1.21.md#kustomize-updates-in-kubectl
For examples and guides for using the kubectl integration please
see the [kubernetes documentation].
see the [kubectl book] or the [kubernetes documentation].
## Usage
@@ -149,9 +134,8 @@ The YAML can be directly [applied] to a cluster:
## Community
- [file a bug](https://kubectl.docs.kubernetes.io/contributing/kustomize/bugs/)
- [contribute a feature](https://kubectl.docs.kubernetes.io/contributing/kustomize/features/)
- [propose a larger enhancement](https://github.com/kubernetes-sigs/kustomize/tree/master/proposals)
- [file a bug](https://kubernetes-sigs.github.io/kustomize/contributing/bugs/) instructions
- [contribute a feature](https://kubernetes-sigs.github.io/kustomize/contributing/features/) instructions
### Code of conduct
@@ -160,22 +144,27 @@ is governed by the [Kubernetes Code of Conduct].
[`make`]: https://www.gnu.org/software/make
[`sed`]: https://www.gnu.org/software/sed
[DAM]: https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#declarative-application-management
[KEP]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cli/2377-Kustomize/README.md
[DAM]: https://kubernetes-sigs.github.io/kustomize/api-reference/glossary#declarative-application-management
[KEP]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cli/0008-kustomize.md
[Kubernetes Code of Conduct]: code-of-conduct.md
[applied]: https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#apply
[base]: https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#base
[declarative configuration]: https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#declarative-application-management
[imageBase]: images/base.jpg
[imageOverlay]: images/overlay.jpg
[applied]: https://kubernetes-sigs.github.io/kustomize/api-reference/glossary#apply
[base]: https://kubernetes-sigs.github.io/kustomize/api-reference/glossary#base
[declarative configuration]: https://kubernetes-sigs.github.io/kustomize/api-reference/glossary#declarative-application-management
[imageBase]: docs/images/base.jpg
[imageOverlay]: docs/images/overlay.jpg
[kubectl announcement]: https://kubernetes.io/blog/2019/03/25/kubernetes-1-14-release-announcement
[kubectl book]: https://kubectl.docs.kubernetes.io/guides/introduction/kustomize/
[kubernetes documentation]: https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/
[kubernetes style]: https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#kubernetes-style-object
[kustomization]: https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#kustomization
[overlay]: https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#overlay
[overlays]: https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#overlay
[kubernetes style]: https://kubernetes-sigs.github.io/kustomize/api-reference/glossary#kubernetes-style-object
[kustomization]: https://kubernetes-sigs.github.io/kustomize/api-reference/glossary#kustomization
[overlay]: https://kubernetes-sigs.github.io/kustomize/api-reference/glossary#overlay
[overlays]: https://kubernetes-sigs.github.io/kustomize/api-reference/glossary#overlay
[release page]: https://github.com/kubernetes-sigs/kustomize/releases
[resource]: https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#resource
[resources]: https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#resource
[resource]: https://kubernetes-sigs.github.io/kustomize/api-reference/glossary#resource
[resources]: https://kubernetes-sigs.github.io/kustomize/api-reference/glossary#resource
[sig-cli]: https://github.com/kubernetes/community/blob/master/sig-cli/README.md
[variants]: https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#variant
[variant]: https://kubernetes-sigs.github.io/kustomize/api-reference/glossary#variant
[variants]: https://kubernetes-sigs.github.io/kustomize/api-reference/glossary#variant
[v2.0.3]: https://github.com/kubernetes-sigs/kustomize/releases/tag/v2.0.3
[v2.1.0]: https://github.com/kubernetes-sigs/kustomize/releases/tag/v2.1.0
[workflows]: https://kubernetes-sigs.github.io/kustomize/guides

View File

@@ -1,112 +0,0 @@
# Kustomize roadmap 2022
Presented at the [January 26, 2022, SIG-CLI meeting](https://youtu.be/l2plzJ9MRlk?t=1321)
kustomize maintainers: @knverey, @natasha41575
[Objective: Improve contributor community](#objective-improve-contributor-community)
[Objective: Improve end-user experience](#objective-improve-end-user-experience)
[Objective: Improve extension experience](#objective-improve-extension-experience)
## Objective: Improve contributor community
**_WHO: End user who also contributes source code._**
Top priority:
- Kustomization v1 (also end-user impact) ([PROJECT](https://github.com/kubernetes-sigs/kustomize/projects/12))
- Remove the following fields:
- [vars](https://github.com/kubernetes-sigs/kustomize/issues/2052)
- [patchesJson6902, patchesStrategicMerge (consolidate on \`patches)](https://github.com/kubernetes-sigs/kustomize/issues/4376)
- [helmChartInflationGenerator, helmCharts, helmGlobals](https://github.com/kubernetes-sigs/kustomize/issues/4401)
- all long-deprecated fields in Kustomization v1 such as \`bases\` and those being accommodate by kustomize edit \[[see code snippet](https://github.com/kubernetes-sigs/kustomize/blob/ee4b7847f0beb6c0d2070673b10f23f7b3e92e82/api/types/fix.go#L15)\]
- Ensure that \`kustomize edit fix\` handles migrations for all those, and that anything it changes is not still present in v1.
- [Add reorder field](https://github.com/kubernetes-sigs/kustomize/issues/3913). Default should be FIFO and legacy should also be supported (could add alphabetic and custom sort support eventually). Replaces -reorder flag.
- [Reconcile openapi and crds field](https://github.com/kubernetes-sigs/kustomize/issues/3944)
- [Consider deprecating configurations field](https://github.com/kubernetes-sigs/kustomize/issues/3945) (old, pre-plugin, pre-openapi global configuration)
- [Add a field to enable the managedby label](https://github.com/kubernetes-sigs/kustomize/issues/4047)
Second priority:
- Improve contributor documentation
- [Instructions to upgrade kustomize-in-kubectl](https://github.com/kubernetes-sigs/kustomize/issues/3951)
Also very valuable to the project:
- [Improve the release process](https://github.com/kubernetes-sigs/kustomize/issues/3952) to support regular biweekly releases [PROJECT](https://github.com/kubernetes-sigs/kustomize/projects/7)
- Release sigs.k8s.io/kustomize/api v1.0.0 [PROJECT](https://github.com/kubernetes-sigs/kustomize/projects/5)
- [Reduce the public surface of the API module](https://github.com/kubernetes-sigs/kustomize/issues/3942)
- [Vendor all transitive deps](https://github.com/kubernetes-sigs/kustomize/issues/3706). Since kustomize is in kubectl, we must do as kubectl does to manage deps, exposing new transitive deps in code review.
- Project administration
- [Rename master branch to main](https://github.com/kubernetes-sigs/kustomize/issues/3954)
## Objective: Improve end-user experience
**_WHO: End user that wants kustomize build artifacts (binaries, containers)._**
Top priorities:
- Bug fixes:
- Fix bugs in basic anchor support: [issue query](https://github.com/kubernetes-sigs/kustomize/issues?q=is%3Aopen+is%3Aissue+label%3Aarea%2Fanchors)
- integer keys support: [#3446](https://github.com/kubernetes-sigs/kustomize/issues/3446)
- kyaml not respecting \`$patch replace|retainKeys\`: [#2037](https://github.com/kubernetes-sigs/kustomize/issues/2037)
- kustomize removing quotes from namespace field values: [#4146](https://github.com/kubernetes-sigs/kustomize/issues/4146)
- Kustomize doesnt support metadata.generateName: [#641](https://github.com/kubernetes-sigs/kustomize/issues/641)
- Send kustomize CLI version number into kubectl ([kubectl issue](https://github.com/kubernetes/kubectl/issues/797) / [kustomize issue](https://github.com/kubernetes-sigs/kustomize/issues/1424))
- Kustomize performance investigations/improvements [PROJECT](https://github.com/kubernetes-sigs/kustomize/projects/13)
- [Support generic resource references in name reference tracking](https://github.com/kubernetes-sigs/kustomize/issues/3418)
- [KEP 4267: retain the resource origin and transformer data in annotations](https://github.com/kubernetes-sigs/kustomize/pull/4267)
Secondary priorities:
- kustomize cli v5 ([PROJECT](https://github.com/kubernetes-sigs/kustomize/projects/14))
- [Drop the --reorder flag](https://github.com/kubernetes-sigs/kustomize/issues/3947)
- [Graduate cfg read-only commands out of alpha](https://github.com/kubernetes-sigs/kustomize/issues/4090).
- [Drop the enable-managedby-label](https://github.com/kubernetes-sigs/kustomize/issues/4047)
- Drop old plugin-related fields in favor of [the Catalog-style fields](https://github.com/kubernetes/enhancements/tree/master/keps/sig-cli/2906-kustomize-function-catalog).
- [Drop the helm flags](https://github.com/kubernetes-sigs/kustomize/issues/4401)
- [Confusion around namespace replacement](https://github.com/kubernetes-sigs/kustomize/issues/880).
Also very valuable to the project:
- [Overinclusion of root directory error in error messages](https://github.com/kubernetes-sigs/kustomize/issues/4348)
- [Add kustomize localize command](https://github.com/kubernetes-sigs/kustomize/issues/3980)
- [Fix Windows support in test suite](https://github.com/kubernetes-sigs/kustomize/issues/4001)
- Improve end-user documentation [PROJECT](https://github.com/kubernetes-sigs/kustomize/projects/9)
## Objective: Improve extension experience
**_WHO: Plugin developers: end users who extend kustomize, but dont think about internals._**
This objective is described in detail in the [Kustomize Plugin Graduation KEP](https://github.com/kubernetes/enhancements/tree/master/keps/sig-cli/2953-kustomize-plugin-graduation) / [PROJECT](https://github.com/kubernetes-sigs/kustomize/projects/15) .
Top priorities:
- Fix core usability issues with KRM Function extensions:
- [Better errors for function config failures](https://github.com/kubernetes-sigs/kustomize/issues/4398)
- [Container KRM Mounts are not mounting via function parameters](https://github.com/kubernetes-sigs/kustomize/issues/4290)
- [Resolution of local file references in extensions transformer configuration](https://github.com/kubernetes-sigs/kustomize/issues/4154)
- [Do not silently ignore plugins when config has typo](https://github.com/kubernetes-sigs/kustomize/issues/4399)
- [KRM Exec Function can't locate executable when referencing a base](https://github.com/kubernetes-sigs/kustomize/issues/4347)
- Once core usability issues are fixed, [deprecate legacy exec and Go plugin support](https://github.com/kubernetes/enhancements/tree/master/keps/sig-cli/2953-kustomize-plugin-graduation)
- [Catalog KEP](https://github.com/kubernetes/enhancements/tree/master/keps/sig-cli/2906-kustomize-function-catalog)
Secondary priorities:
- [Remove Starlark support](https://github.com/kubernetes-sigs/kustomize/issues/4349)
- [Composition KEP](https://github.com/kubernetes/enhancements/pull/2300). The implementation is complete in [#4223](https://github.com/kubernetes-sigs/kustomize/pull/4323), but depends on:
- [Convert resources and components to be backed by a reusable generator](https://github.com/kubernetes-sigs/kustomize/issues/4402)
- [Enable explicitly invoked transformers to use default fieldSpecs](https://github.com/kubernetes-sigs/kustomize/issues/4404)
- [Enable built-in generators to be used in the transformers field ](https://github.com/kubernetes-sigs/kustomize/issues/4403)
Also very valuable to the project:
- [Improve docs for kyaml libraries](https://github.com/kubernetes-sigs/kustomize/issues/3950), especially by adding examples.
- [Create a reserved field for plugin runtime information](https://github.com/kubernetes-sigs/kustomize/issues/4405)
- [Develop new standard process for implementing builtin transformers](https://github.com/kubernetes-sigs/kustomize/issues/4400)

View File

@@ -1,13 +0,0 @@
# Copyright 2022 The Kubernetes Authors.
# SPDX-License-Identifier: Apache-2.0
include ../Makefile-modules.mk
test:
go test -v -timeout 45m -cover ./... -ldflags "-X sigs.k8s.io/kustomize/api/provenance.buildDate=2023-01-31T23:38:41Z -X sigs.k8s.io/kustomize/api/provenance.version=(test)"
build:
go build -ldflags "-X sigs.k8s.io/kustomize/api/provenance.buildDate=$(shell date -u +"%Y-%m-%dT%H:%M:%SZ")" ./...
generate: $(MYGOBIN)/k8scopy $(MYGOBIN)/stringer
go generate ./...

View File

@@ -1,5 +1,5 @@
// Code generated by pluginator on AnnotationsTransformer; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins

View File

@@ -1,5 +1,5 @@
// Code generated by pluginator on ConfigMapGenerator; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins

View File

@@ -1,5 +1,5 @@
// Code generated by pluginator on HashTransformer; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins

View File

@@ -1,11 +1,12 @@
// Code generated by pluginator on HelmChartInflationGenerator; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins
import (
"bytes"
"fmt"
"io/ioutil"
"os"
"os/exec"
"path/filepath"
@@ -13,14 +14,14 @@ import (
"strings"
"github.com/imdario/mergo"
"github.com/pkg/errors"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/errors"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/yaml"
)
// Generate resources from a remote or local helm chart.
// HelmChartInflationGeneratorPlugin is a plugin to generate resources
// from a remote or local helm chart.
type HelmChartInflationGeneratorPlugin struct {
h *resmap.PluginHelpers
types.HelmGlobals
@@ -28,6 +29,8 @@ type HelmChartInflationGeneratorPlugin struct {
tmpDir string
}
var KustomizePlugin HelmChartInflationGeneratorPlugin
const (
valuesMergeOptionMerge = "merge"
valuesMergeOptionOverride = "override"
@@ -70,7 +73,7 @@ func (p *HelmChartInflationGeneratorPlugin) establishTmpDir() (err error) {
// already done.
return nil
}
p.tmpDir, err = os.MkdirTemp("", "kustomize-helm-")
p.tmpDir, err = ioutil.TempDir("", "kustomize-helm-")
return err
}
@@ -84,23 +87,15 @@ func (p *HelmChartInflationGeneratorPlugin) validateArgs() (err error) {
// the loader root (unless root restrictions are
// disabled, in which case this can be an absolute path).
if p.ChartHome == "" {
p.ChartHome = types.HelmDefaultHome
p.ChartHome = "charts"
}
// The ValuesFile(s) may be consulted by the plugin, so it must
// The ValuesFile may be consulted by the plugin, so it must
// be under the loader root (unless root restrictions are
// disabled).
if p.ValuesFile == "" {
p.ValuesFile = filepath.Join(p.ChartHome, p.Name, "values.yaml")
}
for i, file := range p.AdditionalValuesFiles {
// use Load() to enforce root restrictions
if _, err := p.h.Loader().Load(file); err != nil {
return errors.WrapPrefixf(err, "could not load additionalValuesFile")
}
// the additional values filepaths must be relative to the kust root
p.AdditionalValuesFiles[i] = filepath.Join(p.h.Loader().Root(), file)
}
if err = p.errIfIllegalValuesMerge(); err != nil {
return err
@@ -109,7 +104,7 @@ func (p *HelmChartInflationGeneratorPlugin) validateArgs() (err error) {
// ConfigHome is not loaded by the plugin, and can be located anywhere.
if p.ConfigHome == "" {
if err = p.establishTmpDir(); err != nil {
return errors.WrapPrefixf(
return errors.Wrap(
err, "unable to create tmp dir for HELM_CONFIG_HOME")
}
p.ConfigHome = filepath.Join(p.tmpDir, "helm")
@@ -153,10 +148,10 @@ func (p *HelmChartInflationGeneratorPlugin) runHelmCommand(
err := cmd.Run()
if err != nil {
helm := p.h.GeneralConfig().HelmConfig.Command
err = errors.WrapPrefixf(
err = errors.Wrap(
fmt.Errorf(
"unable to run: '%s %s' with env=%s (is '%s' installed?): %w",
helm, strings.Join(args, " "), env, helm, err),
"unable to run: '%s %s' with env=%s (is '%s' installed?)",
helm, strings.Join(args, " "), env, helm),
stderr.String(),
)
}
@@ -216,7 +211,7 @@ func (p *HelmChartInflationGeneratorPlugin) writeValuesBytes(
return "", fmt.Errorf("cannot create tmp dir to write helm values")
}
path := filepath.Join(p.tmpDir, p.Name+"-kustomize-values.yaml")
return path, errors.WrapPrefixf(os.WriteFile(path, b, 0644), "failed to write values file")
return path, ioutil.WriteFile(path, b, 0644)
}
func (p *HelmChartInflationGeneratorPlugin) cleanup() {
@@ -249,31 +244,46 @@ func (p *HelmChartInflationGeneratorPlugin) Generate() (rm resmap.ResMap, err er
return nil, err
}
var stdout []byte
stdout, err = p.runHelmCommand(p.AsHelmArgs(p.absChartHome()))
stdout, err = p.runHelmCommand(p.templateCommand())
if err != nil {
return nil, err
}
rm, resMapErr := p.h.ResmapFactory().NewResMapFromBytes(stdout)
if resMapErr == nil {
rm, err = p.h.ResmapFactory().NewResMapFromBytes(stdout)
if err == nil {
return rm, nil
}
// try to remove the contents before first "---" because
// helm may produce messages to stdout before it
r := &kio.ByteReader{Reader: bytes.NewBufferString(string(stdout)), OmitReaderAnnotations: true}
nodes, err := r.Read()
if err != nil {
return nil, fmt.Errorf("error reading helm output: %w", err)
stdoutStr := string(stdout)
if idx := strings.Index(stdoutStr, "---"); idx != -1 {
return p.h.ResmapFactory().NewResMapFromBytes([]byte(stdoutStr[idx:]))
}
return nil, err
}
if len(nodes) != 0 {
rm, err = p.h.ResmapFactory().NewResMapFromRNodeSlice(nodes)
if err != nil {
return nil, fmt.Errorf("could not parse rnode slice into resource map: %w", err)
}
return rm, nil
func (p *HelmChartInflationGeneratorPlugin) templateCommand() []string {
args := []string{"template"}
if p.ReleaseName != "" {
args = append(args, p.ReleaseName)
}
return nil, fmt.Errorf("could not parse bytes into resource map: %w", resMapErr)
if p.Namespace != "" {
args = append(args, "--namespace", p.Namespace)
}
args = append(args, filepath.Join(p.absChartHome(), p.Name))
if p.ValuesFile != "" {
args = append(args, "--values", p.ValuesFile)
}
if p.ReleaseName == "" {
// AFAICT, this doesn't work as intended due to a bug in helm.
// See https://github.com/helm/helm/issues/6019
// I've tried placing the flag before and after the name argument.
args = append(args, "--generate-name")
}
if p.IncludeCRDs {
args = append(args, "--include-crds")
}
return args
}
func (p *HelmChartInflationGeneratorPlugin) pullCommand() []string {

View File

@@ -1,5 +1,5 @@
// Code generated by pluginator on IAMPolicyGenerator; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins

View File

@@ -1,5 +1,5 @@
// Code generated by pluginator on ImageTagTransformer; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins

View File

@@ -1,5 +1,5 @@
// Code generated by pluginator on LabelTransformer; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins

View File

@@ -0,0 +1,46 @@
// Code generated by pluginator on LegacyOrderTransformer; DO NOT EDIT.
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins
import (
"sort"
"github.com/pkg/errors"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/resource"
)
// Sort the resources using an ordering defined in the Gvk class.
// This puts cluster-wide basic resources with no
// dependencies (like Namespace, StorageClass, etc.)
// first, and resources with a high number of dependencies
// (like ValidatingWebhookConfiguration) last.
type LegacyOrderTransformerPlugin struct{}
// Nothing needed for configuration.
func (p *LegacyOrderTransformerPlugin) Config(
_ *resmap.PluginHelpers, _ []byte) (err error) {
return nil
}
func (p *LegacyOrderTransformerPlugin) Transform(m resmap.ResMap) (err error) {
resources := make([]*resource.Resource, m.Size())
ids := m.AllIds()
sort.Sort(resmap.IdSlice(ids))
for i, id := range ids {
resources[i], err = m.GetByCurrentId(id)
if err != nil {
return errors.Wrap(err, "expected match for sorting")
}
}
m.Clear()
for _, r := range resources {
m.Append(r)
}
return nil
}
func NewLegacyOrderTransformerPlugin() resmap.TransformerPlugin {
return &LegacyOrderTransformerPlugin{}
}

View File

@@ -0,0 +1,55 @@
// Code generated by pluginator on NamespaceTransformer; DO NOT EDIT.
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins
import (
"fmt"
"sigs.k8s.io/kustomize/api/filters/namespace"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/yaml"
)
// Change or set the namespace of non-cluster level resources.
type NamespaceTransformerPlugin struct {
types.ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`
FieldSpecs []types.FieldSpec `json:"fieldSpecs,omitempty" yaml:"fieldSpecs,omitempty"`
}
func (p *NamespaceTransformerPlugin) Config(
_ *resmap.PluginHelpers, c []byte) (err error) {
p.Namespace = ""
p.FieldSpecs = nil
return yaml.Unmarshal(c, p)
}
func (p *NamespaceTransformerPlugin) Transform(m resmap.ResMap) error {
if len(p.Namespace) == 0 {
return nil
}
for _, r := range m.Resources() {
if r.IsNilOrEmpty() {
// Don't mutate empty objects?
continue
}
r.StorePreviousId()
if err := r.ApplyFilter(namespace.Filter{
Namespace: p.Namespace,
FsSlice: p.FieldSpecs,
}); err != nil {
return err
}
matches := m.GetMatchingResourcesByCurrentId(r.CurId().Equals)
if len(matches) != 1 {
return fmt.Errorf(
"namespace transformation produces ID conflict: %+v", matches)
}
}
return nil
}
func NewNamespaceTransformerPlugin() resmap.TransformerPlugin {
return &NamespaceTransformerPlugin{}
}

View File

@@ -1,5 +1,5 @@
// Code generated by pluginator on PatchJson6902Transformer; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins
@@ -7,12 +7,11 @@ import (
"fmt"
jsonpatch "github.com/evanphx/json-patch"
"github.com/pkg/errors"
"sigs.k8s.io/kustomize/api/filters/patchjson6902"
"sigs.k8s.io/kustomize/api/ifc"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/errors"
"sigs.k8s.io/kustomize/kyaml/kio/kioutil"
"sigs.k8s.io/yaml"
)
@@ -61,7 +60,7 @@ func (p *PatchJson6902TransformerPlugin) Config(
}
p.decodedPatch, err = jsonpatch.DecodePatch([]byte(p.JsonOp))
if err != nil {
return errors.WrapPrefixf(err, "decoding %s", p.JsonOp)
return errors.Wrapf(err, "decoding %s", p.JsonOp)
}
if len(p.decodedPatch) == 0 {
return fmt.Errorf(
@@ -79,23 +78,12 @@ func (p *PatchJson6902TransformerPlugin) Transform(m resmap.ResMap) error {
return err
}
for _, res := range resources {
internalAnnotations := kioutil.GetInternalAnnotations(&res.RNode)
err = res.ApplyFilter(patchjson6902.Filter{
Patch: p.JsonOp,
})
if err != nil {
return err
}
annotations := res.GetAnnotations()
for key, value := range internalAnnotations {
annotations[key] = value
}
err = res.SetAnnotations(annotations)
if err != nil {
return err
}
}
return nil
}

View File

@@ -1,5 +1,5 @@
// Code generated by pluginator on PatchStrategicMergeTransformer; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins

View File

@@ -1,5 +1,5 @@
// Code generated by pluginator on PatchTransformer; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins
@@ -12,7 +12,6 @@ import (
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/resource"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio/kioutil"
"sigs.k8s.io/yaml"
)
@@ -112,19 +111,12 @@ func (p *PatchTransformerPlugin) transformJson6902(m resmap.ResMap, patch jsonpa
}
for _, res := range resources {
res.StorePreviousId()
internalAnnotations := kioutil.GetInternalAnnotations(&res.RNode)
err = res.ApplyFilter(patchjson6902.Filter{
Patch: p.Patch,
})
if err != nil {
return err
}
annotations := res.GetAnnotations()
for key, value := range internalAnnotations {
annotations[key] = value
}
err = res.SetAnnotations(annotations)
}
return nil
}

View File

@@ -0,0 +1,103 @@
// Code generated by pluginator on PrefixSuffixTransformer; DO NOT EDIT.
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins
import (
"errors"
"sigs.k8s.io/kustomize/api/filters/prefixsuffix"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/yaml"
)
// Add the given prefix and suffix to the field.
type PrefixSuffixTransformerPlugin struct {
Prefix string `json:"prefix,omitempty" yaml:"prefix,omitempty"`
Suffix string `json:"suffix,omitempty" yaml:"suffix,omitempty"`
FieldSpecs types.FsSlice `json:"fieldSpecs,omitempty" yaml:"fieldSpecs,omitempty"`
}
// A Gvk skip list for prefix/suffix modification.
// hard coded for now - eventually should be part of config.
var prefixSuffixFieldSpecsToSkip = types.FsSlice{
{Gvk: resid.Gvk{Kind: "CustomResourceDefinition"}},
{Gvk: resid.Gvk{Group: "apiregistration.k8s.io", Kind: "APIService"}},
{Gvk: resid.Gvk{Kind: "Namespace"}},
}
func (p *PrefixSuffixTransformerPlugin) Config(
_ *resmap.PluginHelpers, c []byte) (err error) {
p.Prefix = ""
p.Suffix = ""
p.FieldSpecs = nil
err = yaml.Unmarshal(c, p)
if err != nil {
return
}
if p.FieldSpecs == nil {
return errors.New("fieldSpecs is not expected to be nil")
}
return
}
func (p *PrefixSuffixTransformerPlugin) Transform(m resmap.ResMap) error {
// Even if both the Prefix and Suffix are empty we want
// to proceed with the transformation. This allows to add contextual
// information to the resources (AddNamePrefix and AddNameSuffix).
for _, r := range m.Resources() {
// TODO: move this test into the filter (i.e. make a better filter)
if p.shouldSkip(r.OrgId()) {
continue
}
id := r.OrgId()
// current default configuration contains
// only one entry: "metadata/name" with no GVK
for _, fs := range p.FieldSpecs {
// TODO: this is redundant to filter (but needed for now)
if !id.IsSelected(&fs.Gvk) {
continue
}
// TODO: move this test into the filter.
if smellsLikeANameChange(&fs) {
// "metadata/name" is the only field.
// this will add a prefix and a suffix
// to the resource even if those are
// empty
r.AddNamePrefix(p.Prefix)
r.AddNameSuffix(p.Suffix)
if p.Prefix != "" || p.Suffix != "" {
r.StorePreviousId()
}
}
if err := r.ApplyFilter(prefixsuffix.Filter{
Prefix: p.Prefix,
Suffix: p.Suffix,
FieldSpec: fs,
}); err != nil {
return err
}
}
}
return nil
}
func smellsLikeANameChange(fs *types.FieldSpec) bool {
return fs.Path == "metadata/name"
}
func (p *PrefixSuffixTransformerPlugin) shouldSkip(id resid.ResId) bool {
for _, path := range prefixSuffixFieldSpecsToSkip {
if id.IsSelected(&path.Gvk) {
return true
}
}
return false
}
func NewPrefixSuffixTransformerPlugin() resmap.TransformerPlugin {
return &PrefixSuffixTransformerPlugin{}
}

View File

@@ -0,0 +1,59 @@
// Code generated by pluginator on ReplacementTransformer; DO NOT EDIT.
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins
import (
"fmt"
"sigs.k8s.io/kustomize/api/filters/replacement"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/yaml"
)
// Replace values in targets with values from a source
type ReplacementTransformerPlugin struct {
ReplacementList []types.ReplacementField `json:"replacements,omitempty" yaml:"replacements,omitempty"`
Replacements []types.Replacement `json:"omitempty" yaml:"omitempty"`
}
func (p *ReplacementTransformerPlugin) Config(
h *resmap.PluginHelpers, c []byte) (err error) {
p.ReplacementList = []types.ReplacementField{}
if err := yaml.Unmarshal(c, p); err != nil {
return err
}
for _, r := range p.ReplacementList {
if r.Path != "" && (r.Source != nil || len(r.Targets) != 0) {
return fmt.Errorf("cannot specify both path and inline replacement")
}
if r.Path != "" {
// load the replacement from the path
content, err := h.Loader().Load(r.Path)
if err != nil {
return err
}
repl := types.Replacement{}
if err := yaml.Unmarshal(content, &repl); err != nil {
return err
}
p.Replacements = append(p.Replacements, repl)
} else {
// replacement information is already loaded
p.Replacements = append(p.Replacements, r.Replacement)
}
}
return nil
}
func (p *ReplacementTransformerPlugin) Transform(m resmap.ResMap) (err error) {
return m.ApplyFilter(replacement.Filter{
Replacements: p.Replacements,
})
}
func NewReplacementTransformerPlugin() resmap.TransformerPlugin {
return &ReplacementTransformerPlugin{}
}

View File

@@ -1,5 +1,5 @@
// Code generated by pluginator on ReplicaCountTransformer; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins

View File

@@ -1,5 +1,5 @@
// Code generated by pluginator on SecretGenerator; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins

View File

@@ -1,5 +1,5 @@
// Code generated by pluginator on ValueAddTransformer; DO NOT EDIT.
// pluginator {(devel) unknown }
// pluginator {unknown 1970-01-01T00:00:00Z }
package builtins

View File

@@ -1,49 +0,0 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Deprecated: Package api/builtins will not be available in API v1.
package builtins
import (
internal "sigs.k8s.io/kustomize/api/internal/builtins"
)
type (
AnnotationsTransformerPlugin = internal.AnnotationsTransformerPlugin
ConfigMapGeneratorPlugin = internal.ConfigMapGeneratorPlugin
HashTransformerPlugin = internal.HashTransformerPlugin
HelmChartInflationGeneratorPlugin = internal.HelmChartInflationGeneratorPlugin
IAMPolicyGeneratorPlugin = internal.IAMPolicyGeneratorPlugin
ImageTagTransformerPlugin = internal.ImageTagTransformerPlugin
LabelTransformerPlugin = internal.LabelTransformerPlugin
NamespaceTransformerPlugin = internal.NamespaceTransformerPlugin
PatchJson6902TransformerPlugin = internal.PatchJson6902TransformerPlugin
PatchStrategicMergeTransformerPlugin = internal.PatchStrategicMergeTransformerPlugin
PatchTransformerPlugin = internal.PatchTransformerPlugin
PrefixTransformerPlugin = internal.PrefixTransformerPlugin
SuffixTransformerPlugin = internal.SuffixTransformerPlugin
ReplacementTransformerPlugin = internal.ReplacementTransformerPlugin
ReplicaCountTransformerPlugin = internal.ReplicaCountTransformerPlugin
SecretGeneratorPlugin = internal.SecretGeneratorPlugin
ValueAddTransformerPlugin = internal.ValueAddTransformerPlugin
)
var (
NewAnnotationsTransformerPlugin = internal.NewAnnotationsTransformerPlugin
NewConfigMapGeneratorPlugin = internal.NewConfigMapGeneratorPlugin
NewHashTransformerPlugin = internal.NewHashTransformerPlugin
NewHelmChartInflationGeneratorPlugin = internal.NewHelmChartInflationGeneratorPlugin
NewIAMPolicyGeneratorPlugin = internal.NewIAMPolicyGeneratorPlugin
NewImageTagTransformerPlugin = internal.NewImageTagTransformerPlugin
NewLabelTransformerPlugin = internal.NewLabelTransformerPlugin
NewNamespaceTransformerPlugin = internal.NewNamespaceTransformerPlugin
NewPatchJson6902TransformerPlugin = internal.NewPatchJson6902TransformerPlugin
NewPatchStrategicMergeTransformerPlugin = internal.NewPatchStrategicMergeTransformerPlugin
NewPatchTransformerPlugin = internal.NewPatchTransformerPlugin
NewPrefixTransformerPlugin = internal.NewPrefixTransformerPlugin
NewSuffixTransformerPlugin = internal.NewSuffixTransformerPlugin
NewReplacementTransformerPlugin = internal.NewReplacementTransformerPlugin
NewReplicaCountTransformerPlugin = internal.NewReplicaCountTransformerPlugin
NewSecretGeneratorPlugin = internal.NewSecretGeneratorPlugin
NewValueAddTransformerPlugin = internal.NewValueAddTransformerPlugin
)

View File

@@ -4,7 +4,7 @@
package filesys
import (
"os"
"io/ioutil"
"path/filepath"
"strings"
)
@@ -17,12 +17,12 @@ type ConfirmedDir string
// The directory is cleaned, no symlinks, etc. so it's
// returned as a ConfirmedDir.
func NewTmpConfirmedDir() (ConfirmedDir, error) {
n, err := os.MkdirTemp("", "kustomize-")
n, err := ioutil.TempDir("", "kustomize-")
if err != nil {
return "", err
}
// In MacOs `os.MkdirTemp` creates a directory
// In MacOs `ioutil.TempDir` creates a directory
// with root in the `/var` folder, which is in turn
// a symlinked path to `/private/var`.
// Function `filepath.EvalSymlinks`is used to

View File

@@ -1,20 +1,20 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
//go:build !windows
// +build !windows
package filesys
package filesys_test
import (
"os"
"path/filepath"
"testing"
. "sigs.k8s.io/kustomize/api/filesys"
)
func TestJoin(t *testing.T) {
fSys := MakeFsInMemory()
if err := fSys.Mkdir("/foo"); err != nil {
err := fSys.Mkdir("/foo")
if err != nil {
t.Fatalf("unexpected err: %v", err)
}
d, f, err := fSys.CleanedAbs("/foo")

View File

@@ -1,61 +0,0 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package filesys provides a file system abstraction,
// a subset of that provided by golang.org/pkg/os,
// with an on-disk and in-memory representation.
//
// Deprecated: use sigs.k8s.io/kustomize/kyaml/filesys instead.
package filesys
import "sigs.k8s.io/kustomize/kyaml/filesys"
const (
// Separator is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.Separator.
Separator = filesys.Separator
// SelfDir is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.SelfDir.
SelfDir = filesys.SelfDir
// ParentDir is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.ParentDir.
ParentDir = filesys.ParentDir
)
type (
// FileSystem is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.FileSystem.
FileSystem = filesys.FileSystem
// FileSystemOrOnDisk is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.FileSystemOrOnDisk.
FileSystemOrOnDisk = filesys.FileSystemOrOnDisk
// ConfirmedDir is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.ConfirmedDir.
ConfirmedDir = filesys.ConfirmedDir
)
// MakeEmptyDirInMemory is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.MakeEmptyDirInMemory.
func MakeEmptyDirInMemory() FileSystem { return filesys.MakeEmptyDirInMemory() }
// MakeFsInMemory is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.MakeFsInMemory.
func MakeFsInMemory() FileSystem { return filesys.MakeFsInMemory() }
// MakeFsOnDisk is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.MakeFsOnDisk.
func MakeFsOnDisk() FileSystem { return filesys.MakeFsOnDisk() }
// NewTmpConfirmedDir is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.NewTmpConfirmedDir.
func NewTmpConfirmedDir() (filesys.ConfirmedDir, error) { return filesys.NewTmpConfirmedDir() }
// RootedPath is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.RootedPath.
func RootedPath(elem ...string) string { return filesys.RootedPath(elem...) }
// StripTrailingSeps is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.StripTrailingSeps.
func StripTrailingSeps(s string) string { return filesys.StripTrailingSeps(s) }
// StripLeadingSeps is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.StripLeadingSeps.
func StripLeadingSeps(s string) string { return filesys.StripLeadingSeps(s) }
// PathSplit is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.PathSplit.
func PathSplit(incoming string) []string { return filesys.PathSplit(incoming) }
// PathJoin is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.PathJoin.
func PathJoin(incoming []string) string { return filesys.PathJoin(incoming) }
// InsertPathPart is deprecated, use sigs.k8s.io/kustomize/kyaml/filesys.InsertPathPart.
func InsertPathPart(path string, pos int, part string) string {
return filesys.InsertPathPart(path, pos, part)
}

51
api/filesys/filesystem.go Normal file
View File

@@ -0,0 +1,51 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package filesys
import (
"path/filepath"
)
const (
Separator = string(filepath.Separator)
SelfDir = "."
ParentDir = ".."
)
// FileSystem groups basic os filesystem methods.
// It's supposed be functional subset of https://golang.org/pkg/os
type FileSystem interface {
// Create a file.
Create(path string) (File, error)
// MkDir makes a directory.
Mkdir(path string) error
// MkDirAll makes a directory path, creating intervening directories.
MkdirAll(path string) error
// RemoveAll removes path and any children it contains.
RemoveAll(path string) error
// Open opens the named file for reading.
Open(path string) (File, error)
// IsDir returns true if the path is a directory.
IsDir(path string) bool
// ReadDir returns a list of files and directories within a directory.
ReadDir(path string) ([]string, error)
// CleanedAbs converts the given path into a
// directory and a file name, where the directory
// is represented as a ConfirmedDir and all that implies.
// If the entire path is a directory, the file component
// is an empty string.
CleanedAbs(path string) (ConfirmedDir, string, error)
// Exists is true if the path exists in the file system.
Exists(path string) bool
// Glob returns the list of matching files,
// emulating https://golang.org/pkg/path/filepath/#Glob
Glob(pattern string) ([]string, error)
// ReadFile returns the contents of the file at the given path.
ReadFile(path string) ([]byte, error)
// WriteFile writes the data to a file at the given path,
// overwriting anything that's already there.
WriteFile(path string, data []byte) error
// Walk walks the file system with the given WalkFunc.
Walk(path string, walkFn filepath.WalkFunc) error
}

View File

@@ -14,7 +14,7 @@ import (
"sort"
"strings"
"sigs.k8s.io/kustomize/kyaml/errors"
"github.com/pkg/errors"
)
var _ File = &fsNode{}
@@ -123,11 +123,11 @@ func (n *fsNode) addFile(name string, c []byte) (result *fsNode, err error) {
if result.offset != nil {
return nil, fmt.Errorf("cannot add already opened file '%s'", n.Path())
}
result.content = append(result.content[:0], c...)
result.content = c
return result, nil
}
result = &fsNode{
content: append([]byte(nil), c...),
content: c,
parent: parent,
}
parent.dir[fileName] = result
@@ -163,6 +163,7 @@ func (n *fsNode) AddFile(
}
func (n *fsNode) addDir(path string) (result *fsNode, err error) {
parent := n
dName, subDirName := mySplit(path)
if dName != "" {
@@ -232,10 +233,10 @@ func (n *fsNode) AddDir(path string) (result *fsNode, err error) {
func (n *fsNode) CleanedAbs(path string) (ConfirmedDir, string, error) {
node, err := n.Find(path)
if err != nil {
return "", "", errors.WrapPrefixf(err, "unable to clean")
return "", "", errors.Wrap(err, "unable to clean")
}
if node == nil {
return "", "", notExistError(path)
return "", "", fmt.Errorf("'%s' doesn't exist", path)
}
if node.isNodeADir() {
return ConfirmedDir(node.Path()), "", nil
@@ -308,8 +309,7 @@ func (n *fsNode) RemoveAll(path string) error {
return err
}
if result == nil {
// If the path doesn't exist, no need to remove anything.
return nil
return fmt.Errorf("cannot find '%s' to remove it", path)
}
return result.Remove()
}
@@ -351,9 +351,6 @@ func (n *fsNode) IsDir(path string) bool {
// ReadDir implements FileSystem.
func (n *fsNode) ReadDir(path string) ([]string, error) {
if !n.Exists(path) {
return nil, notExistError(path)
}
if !n.IsDir(path) {
return nil, fmt.Errorf("%s is not a directory", path)
}
@@ -401,7 +398,7 @@ func (n *fsNode) Open(path string) (File, error) {
return nil, err
}
if result == nil {
return nil, notExistError(path)
return nil, fmt.Errorf("cannot find '%s' to open it", path)
}
if result.offset != nil {
return nil, fmt.Errorf("cannot open previously opened file '%s'", path)
@@ -426,7 +423,7 @@ func (n *fsNode) ReadFile(path string) (c []byte, err error) {
return nil, err
}
if result == nil {
return nil, notExistError(path)
return nil, fmt.Errorf("cannot find '%s' to read it", path)
}
if result.isNodeADir() {
return nil, fmt.Errorf("cannot read content from non-file '%s'", n.Path())
@@ -493,7 +490,7 @@ func (n *fsNode) Walk(path string, walkFn filepath.WalkFunc) error {
return err
}
if result == nil {
return notExistError(path)
return fmt.Errorf("cannot find '%s' to walk it", path)
}
return result.WalkMe(walkFn)
}
@@ -569,7 +566,7 @@ func (n *fsNode) DebugPrint() {
})
}
var legalFileNamePattern = regexp.MustCompile("^[a-zA-Z0-9-_.:]+$")
var legalFileNamePattern = regexp.MustCompile("^[a-zA-Z0-9-_.]+$")
// This rules enforced here should be simpler and tighter
// than what's allowed on a real OS.
@@ -611,7 +608,6 @@ func (n *fsNode) RegExpGlob(pattern string) ([]string, error) {
// This is how /bin/ls behaves.
func (n *fsNode) Glob(pattern string) ([]string, error) {
var result []string
var allFiles []string
err := n.WalkMe(func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
@@ -622,7 +618,7 @@ func (n *fsNode) Glob(pattern string) ([]string, error) {
return err
}
if match {
allFiles = append(allFiles, path)
result = append(result, path)
}
}
return nil
@@ -630,18 +626,6 @@ func (n *fsNode) Glob(pattern string) ([]string, error) {
if err != nil {
return nil, err
}
if IsHiddenFilePath(pattern) {
result = allFiles
} else {
result = RemoveHiddenFiles(allFiles)
}
sort.Strings(result)
return result, nil
}
// notExistError indicates that a file or directory does not exist.
// Unwrapping returns os.ErrNotExist so errors.Is(err, os.ErrNotExist) works correctly.
type notExistError string
func (err notExistError) Error() string { return fmt.Sprintf("'%s' doesn't exist", string(err)) }
func (err notExistError) Unwrap() error { return os.ErrNotExist }

View File

@@ -1,22 +1,18 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
//go:build !windows
// +build !windows
package filesys
import (
"fmt"
"io"
"io/ioutil"
"math/rand"
"os"
"path/filepath"
"sort"
"strings"
"testing"
"github.com/stretchr/testify/require"
)
const content = `
@@ -33,12 +29,6 @@ var topCases = []pathCase{
arg: ParentDir,
errStr: "illegal name '..' in file creation",
},
{
what: "colon",
arg: "a:b",
name: "a:b",
path: "a:b",
},
{
what: "empty",
arg: "",
@@ -100,10 +90,10 @@ func TestMakeFsInMemory(t *testing.T) {
t, "MakeFsInMemory", true, topCases, MakeFsInMemory())
}
//nolint:gocyclo
func runBasicOperations(
t *testing.T, tName string, isFSysRooted bool,
cases []pathCase, fSys FileSystem) {
t.Helper()
for _, c := range cases {
err := fSys.WriteFile(c.arg, []byte(content))
if c.errStr != "" {
@@ -140,7 +130,7 @@ func runBasicOperations(
if fi.Name() != c.name {
t.Fatalf("%s; expected name '%s', got '%s'", c.what, c.name, fi.Name())
}
buff, err := io.ReadAll(f)
buff, err := ioutil.ReadAll(f)
if err != nil {
t.Fatalf("%s; unexpected error: %v", c.what, err)
}
@@ -165,13 +155,9 @@ func runBasicOperations(
if string(stuff) != both {
t.Fatalf("%s; unexpected content '%s', expected '%s'", c.what, stuff, both)
}
content := []byte(shortContent)
if err := fSys.WriteFile(c.path, content); err != nil {
if err := fSys.WriteFile(c.path, []byte(shortContent)); err != nil {
t.Fatalf("%s; unexpected error: %v", c.what, err)
}
// This ensures that modifying the original slice does not change the contents of the file.
content[0] = '@'
stuff, err = fSys.ReadFile(c.path)
if err != nil {
t.Fatalf("%s; unexpected error: %v", c.what, err)
@@ -378,7 +364,6 @@ func TestAddFile(t *testing.T) {
func checkNode(
t *testing.T, what string, f *fsNode, name string,
size int, isDir bool, path string) {
t.Helper()
if f.isNodeADir() != isDir {
t.Fatalf("%s; unexpected isNodeADir = %v", what, f.isNodeADir())
}
@@ -396,7 +381,6 @@ func checkNode(
func checkOsStat(
t *testing.T, what string, f File, name string,
size int, isDir bool) {
t.Helper()
info, err := f.Stat()
if err != nil {
t.Fatalf("%s; unexpected stat error %v", what, err)
@@ -458,10 +442,10 @@ var bunchOfFiles = []struct {
addAsDir: true,
},
{
path: "x",
path: filepath.Join("x"),
},
{
path: "y",
path: filepath.Join("y"),
},
{
path: filepath.Join("b", "d", "a", "c", "i", "beans"),
@@ -473,17 +457,9 @@ var bunchOfFiles = []struct {
{
path: filepath.Join("b", "d", "a", "c", "u"),
},
{
path: filepath.Join("b", "d", ".hidden_file"),
},
{
path: filepath.Join("b", "d", ".hidden_dir"),
addAsDir: true,
},
}
func makeLoadedFileTree(t *testing.T) *fsNode {
t.Helper()
n := MakeEmptyDirInMemory()
var err error
expectedFileCount := 0
@@ -498,7 +474,8 @@ func makeLoadedFileTree(t *testing.T) *fsNode {
t.Fatalf("unexpected error %v", err)
}
}
if fc := n.FileCount(); fc != expectedFileCount {
fc := n.FileCount()
if fc != expectedFileCount {
t.Fatalf("expected file count %d, got %d",
expectedFileCount, fc)
}
@@ -568,14 +545,15 @@ func TestRemove(t *testing.T) {
orgCount -= 3
// Now drop one more for a total of four dropped.
result, _ = n.Find("y")
result, _ = n.Find(filepath.Join("y"))
err = result.Remove()
if err != nil {
t.Fatalf("%s; unable to remove: %v", path, err)
}
orgCount -= 1
if fc := n.FileCount(); fc != orgCount {
fc := n.FileCount()
if fc != orgCount {
t.Fatalf("expected file count %d, got %d",
orgCount, fc)
}
@@ -595,7 +573,6 @@ func TestExists(t *testing.T) {
func TestRegExpGlob(t *testing.T) {
n := makeLoadedFileTree(t)
expected := []string{
filepath.Join("b", "d", ".hidden_file"),
filepath.Join("b", "d", "a", "c", "i", "beans"),
filepath.Join("b", "d", "a", "c", "m"),
filepath.Join("b", "d", "a", "c", "u"),
@@ -615,36 +592,16 @@ func TestRegExpGlob(t *testing.T) {
func TestGlob(t *testing.T) {
n := makeLoadedFileTree(t)
tests := map[string]struct {
globPattern string
expectedFiles []string
}{
"VisibleFiles": {
globPattern: "b/d/*",
expectedFiles: []string{
filepath.Join("b", "d", "x"),
filepath.Join("b", "d", "y"),
filepath.Join("b", "d", "z"),
},
},
"HiddenFiles": {
globPattern: "b/d/.*",
expectedFiles: []string{
filepath.Join("b", "d", ".hidden_file"),
},
},
expected := []string{
filepath.Join("b", "d", "x"),
filepath.Join("b", "d", "y"),
filepath.Join("b", "d", "z"),
}
for test, c := range tests {
t.Run(test, func(t *testing.T) {
paths, err := n.Glob(c.globPattern)
if err != nil {
t.Fatalf("glob error: %v", err)
}
assertEqualStringSlices(t, c.expectedFiles, paths, "glob test")
})
paths, err := n.Glob("b/d/*")
if err != nil {
t.Fatalf("glob error: %v", err)
}
assertEqualStringSlices(t, expected, paths, "glob test")
}
func assertEqualStringSlices(t *testing.T, expected, actual []string, message string) {
@@ -700,7 +657,7 @@ func TestFind(t *testing.T) {
},
{
what: "directory",
arg: "b",
arg: filepath.Join("b"),
expectDir: true,
},
{
@@ -847,32 +804,6 @@ func TestCleanedAbs(t *testing.T) {
}
}
func TestConfirmDirMemRoot(t *testing.T) {
fSys := MakeFsInMemory()
actual, err := ConfirmDir(fSys, Separator)
require.NoError(t, err)
require.Equal(t, Separator, actual.String())
}
func TestConfirmDirRelativeNode(t *testing.T) {
req := require.New(t)
fSysEmpty := MakeEmptyDirInMemory()
fSysRoot, err := fSysEmpty.AddDir("a")
req.NoError(err)
fSysSub, err := fSysRoot.AddDir("b")
req.NoError(err)
err = fSysSub.Mkdir("c")
req.NoError(err)
expected := filepath.Join("a", "b", "c")
req.Truef(fSysEmpty.Exists(expected), existMsg, expected)
actual, err := ConfirmDir(fSysSub, "c")
req.NoError(err)
req.Equal(expected, actual.String())
}
func TestFileOps(t *testing.T) {
const path = "foo.txt"
content := strings.Repeat("longest content", 100)
@@ -903,7 +834,7 @@ func TestFileOps(t *testing.T) {
defer f.Close()
for {
buf := make([]byte, rand.Intn(10)) //nolint:gosec
buf := make([]byte, rand.Intn(10))
n, err := f.Read(buf)
if err != nil && err != io.EOF {
t.Fatalf("unexpected error: %v", err)

View File

@@ -5,11 +5,10 @@ package filesys
import (
"fmt"
"io/ioutil"
"log"
"os"
"path/filepath"
"sigs.k8s.io/kustomize/kyaml/errors"
)
var _ FileSystem = fsOnDisk{}
@@ -58,7 +57,7 @@ func (x fsOnDisk) CleanedAbs(
deLinked, err := filepath.EvalSymlinks(absRoot)
if err != nil {
return "", "", fmt.Errorf(
"evalsymlink failure on '%s' : %w", path, err)
"evalsymlink failure on '%s' : %v", path, err)
}
if x.IsDir(deLinked) {
return ConfirmedDir(deLinked), "", nil
@@ -89,17 +88,7 @@ func (fsOnDisk) Exists(name string) bool {
// Glob returns the list of matching files
func (fsOnDisk) Glob(pattern string) ([]string, error) {
var result []string
allFilePaths, err := filepath.Glob(pattern)
if err != nil {
return nil, err
}
if IsHiddenFilePath(pattern) {
result = allFilePaths
} else {
result = RemoveHiddenFiles(allFilePaths)
}
return result, nil
return filepath.Glob(pattern)
}
// IsDir delegates to os.Stat and FileInfo.IsDir
@@ -124,15 +113,12 @@ func (fsOnDisk) ReadDir(name string) ([]string, error) {
return result, nil
}
// ReadFile delegates to os.ReadFile.
func (fsOnDisk) ReadFile(name string) ([]byte, error) {
content, err := os.ReadFile(name)
return content, errors.Wrap(err)
}
// ReadFile delegates to ioutil.ReadFile.
func (fsOnDisk) ReadFile(name string) ([]byte, error) { return ioutil.ReadFile(name) }
// WriteFile delegates to os.WriteFile with read/write permissions.
// WriteFile delegates to ioutil.WriteFile with read/write permissions.
func (fsOnDisk) WriteFile(name string, c []byte) error {
return errors.Wrap(os.WriteFile(name, c, 0666)) //nolint:gosec
return ioutil.WriteFile(name, c, 0666)
}
// Walk delegates to filepath.Walk.

View File

@@ -0,0 +1,165 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package filesys_test
import (
"io/ioutil"
"os"
"path"
"path/filepath"
"reflect"
"testing"
. "sigs.k8s.io/kustomize/api/filesys"
)
func makeTestDir(t *testing.T) (FileSystem, string) {
fSys := MakeFsOnDisk()
td, err := ioutil.TempDir("", "kustomize_testing_dir")
if err != nil {
t.Fatalf("unexpected error %s", err)
}
testDir, err := filepath.EvalSymlinks(td)
if err != nil {
t.Fatalf("unexpected error %s", err)
}
if !fSys.Exists(testDir) {
t.Fatalf("expected existence")
}
if !fSys.IsDir(testDir) {
t.Fatalf("expected directory")
}
return fSys, testDir
}
func TestCleanedAbs_1(t *testing.T) {
fSys, testDir := makeTestDir(t)
defer os.RemoveAll(testDir)
d, f, err := fSys.CleanedAbs("")
if err != nil {
t.Fatalf("unexpected err=%v", err)
}
wd, err := os.Getwd()
if err != nil {
t.Fatalf("unexpected err=%v", err)
}
if d.String() != wd {
t.Fatalf("unexpected d=%s", d)
}
if f != "" {
t.Fatalf("unexpected f=%s", f)
}
}
func TestCleanedAbs_2(t *testing.T) {
fSys, testDir := makeTestDir(t)
defer os.RemoveAll(testDir)
d, f, err := fSys.CleanedAbs("/")
if err != nil {
t.Fatalf("unexpected err=%v", err)
}
if d != "/" {
t.Fatalf("unexpected d=%s", d)
}
if f != "" {
t.Fatalf("unexpected f=%s", f)
}
}
func TestCleanedAbs_3(t *testing.T) {
fSys, testDir := makeTestDir(t)
defer os.RemoveAll(testDir)
err := fSys.WriteFile(
filepath.Join(testDir, "foo"), []byte(`foo`))
if err != nil {
t.Fatalf("unexpected err=%v", err)
}
d, f, err := fSys.CleanedAbs(filepath.Join(testDir, "foo"))
if err != nil {
t.Fatalf("unexpected err=%v", err)
}
if d.String() != testDir {
t.Fatalf("unexpected d=%s", d)
}
if f != "foo" {
t.Fatalf("unexpected f=%s", f)
}
}
func TestCleanedAbs_4(t *testing.T) {
fSys, testDir := makeTestDir(t)
defer os.RemoveAll(testDir)
err := fSys.MkdirAll(filepath.Join(testDir, "d1", "d2"))
if err != nil {
t.Fatalf("unexpected err=%v", err)
}
err = fSys.WriteFile(
filepath.Join(testDir, "d1", "d2", "bar"),
[]byte(`bar`))
if err != nil {
t.Fatalf("unexpected err=%v", err)
}
d, f, err := fSys.CleanedAbs(
filepath.Join(testDir, "d1", "d2"))
if err != nil {
t.Fatalf("unexpected err=%v", err)
}
if d.String() != filepath.Join(testDir, "d1", "d2") {
t.Fatalf("unexpected d=%s", d)
}
if f != "" {
t.Fatalf("unexpected f=%s", f)
}
d, f, err = fSys.CleanedAbs(
filepath.Join(testDir, "d1", "d2", "bar"))
if err != nil {
t.Fatalf("unexpected err=%v", err)
}
if d.String() != filepath.Join(testDir, "d1", "d2") {
t.Fatalf("unexpected d=%s", d)
}
if f != "bar" {
t.Fatalf("unexpected f=%s", f)
}
}
func TestReadFilesRealFS(t *testing.T) {
fSys, testDir := makeTestDir(t)
defer os.RemoveAll(testDir)
err := fSys.WriteFile(path.Join(testDir, "foo"), []byte(`foo`))
if err != nil {
t.Fatalf("unexpected error %s", err)
}
if !fSys.Exists(path.Join(testDir, "foo")) {
t.Fatalf("expected foo")
}
if fSys.IsDir(path.Join(testDir, "foo")) {
t.Fatalf("expected foo not to be a directory")
}
err = fSys.WriteFile(path.Join(testDir, "bar"), []byte(`bar`))
if err != nil {
t.Fatalf("unexpected error %s", err)
}
files, err := fSys.Glob(path.Join("testDir", "*"))
expected := []string{
path.Join(testDir, "bar"),
path.Join(testDir, "foo"),
}
if err != nil {
t.Fatalf("expected no error")
}
if reflect.DeepEqual(files, expected) {
t.Fatalf("incorrect files found by glob: %v", files)
}
}

125
api/filesys/util.go Normal file
View File

@@ -0,0 +1,125 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package filesys
import (
"os"
"path/filepath"
"strings"
)
// RootedPath returns a rooted path, e.g. "/foo/bar" as
// opposed to "foo/bar".
func RootedPath(elem ...string) string {
return Separator + filepath.Join(elem...)
}
// StripTrailingSeps trims trailing filepath separators from input.
func StripTrailingSeps(s string) string {
k := len(s)
for k > 0 && s[k-1] == filepath.Separator {
k--
}
return s[:k]
}
// StripLeadingSeps trims leading filepath separators from input.
func StripLeadingSeps(s string) string {
k := 0
for k < len(s) && s[k] == filepath.Separator {
k++
}
return s[k:]
}
// PathSplit converts a file path to a slice of string.
// If the path is absolute (if the path has a leading slash),
// then the first entry in the result is an empty string.
// Desired: path == PathJoin(PathSplit(path))
func PathSplit(incoming string) []string {
if incoming == "" {
return []string{}
}
dir, path := filepath.Split(incoming)
if dir == string(os.PathSeparator) {
if path == "" {
return []string{""}
}
return []string{"", path}
}
dir = strings.TrimSuffix(dir, string(os.PathSeparator))
if dir == "" {
return []string{path}
}
return append(PathSplit(dir), path)
}
// PathJoin converts a slice of string to a file path.
// If the first entry is an empty string, then the returned
// path is absolute (it has a leading slash).
// Desired: path == PathJoin(PathSplit(path))
func PathJoin(incoming []string) string {
if len(incoming) == 0 {
return ""
}
if incoming[0] == "" {
return string(os.PathSeparator) + filepath.Join(incoming[1:]...)
}
return filepath.Join(incoming...)
}
// InsertPathPart inserts 'part' at position 'pos' in the given filepath.
// The first position is 0.
//
// E.g. if part == 'PEACH'
//
// OLD : NEW : POS
// --------------------------------------------------------
// {empty} : PEACH : irrelevant
// / : /PEACH : irrelevant
// pie : PEACH/pie : 0 (or negative)
// /pie : /PEACH/pie : 0 (or negative)
// raw : raw/PEACH : 1 (or larger)
// /raw : /raw/PEACH : 1 (or larger)
// a/nice/warm/pie : a/nice/warm/PEACH/pie : 3
// /a/nice/warm/pie : /a/nice/warm/PEACH/pie : 3
//
// * An empty part results in no change.
//
// * Absolute paths get their leading '/' stripped, treated like
// relative paths, and the leading '/' is re-added on output.
// The meaning of pos is intentionally the same in either absolute or
// relative paths; if it weren't, this function could convert absolute
// paths to relative paths, which is not desirable.
//
// * For robustness (liberal input, conservative output) Pos values that
// that are too small (large) to index the split filepath result in a
// prefix (postfix) rather than an error. Use extreme position values
// to assure a prefix or postfix (e.g. 0 will always prefix, and
// 9999 will presumably always postfix).
func InsertPathPart(path string, pos int, part string) string {
if part == "" {
return path
}
parts := PathSplit(path)
if pos < 0 {
pos = 0
} else if pos > len(parts) {
pos = len(parts)
}
if len(parts) > 0 && parts[0] == "" && pos < len(parts) {
// An empty string at 0 indicates an absolute path, and means
// we must increment pos. This change means that a position
// specification has the same meaning in relative and absolute paths.
// E.g. in either the path 'a/b/c' or the path '/a/b/c',
// 'a' is at 0, 'b' is at 1 and 'c' is at 2, and inserting at
// zero means a new first field _without_ changing an absolute
// path to a relative path.
pos++
}
result := make([]string, len(parts)+1)
copy(result, parts[0:pos])
result[pos] = part
return PathJoin(append(result, parts[pos:]...))
}

374
api/filesys/util_test.go Normal file
View File

@@ -0,0 +1,374 @@
package filesys_test
import (
"os"
"path/filepath"
"testing"
. "sigs.k8s.io/kustomize/api/filesys"
)
// Confirm behavior of filepath.Match
func TestFilePathMatch(t *testing.T) {
cases := []struct {
pattern string
path string
expected bool
}{
{
pattern: "*e*",
path: "hey",
expected: true,
},
{
pattern: "*e*",
path: "hay",
expected: false,
},
{
pattern: "*e*",
path: filepath.Join("h", "e", "y"),
expected: false,
},
{
pattern: "*/e/*",
path: filepath.Join("h", "e", "y"),
expected: true,
},
{
pattern: "h/e/*",
path: filepath.Join("h", "e", "y"),
expected: true,
},
{
pattern: "*/e/y",
path: filepath.Join("h", "e", "y"),
expected: true,
},
{
pattern: "*/*/*",
path: filepath.Join("h", "e", "y"),
expected: true,
},
{
pattern: "*/*/*",
path: filepath.Join("h", "e", "y", "there"),
expected: false,
},
{
pattern: "*/*/*/t*e",
path: filepath.Join("h", "e", "y", "there"),
expected: true,
},
}
for _, item := range cases {
match, err := filepath.Match(item.pattern, item.path)
if err != nil {
t.Fatalf("unexpected err: %v", err)
}
if match != item.expected {
t.Fatalf("'%s' '%s' %v\n", item.pattern, item.path, match)
}
}
}
// Confirm behavior of filepath.Split
func TestFilePathSplit(t *testing.T) {
cases := []struct {
full string
dir string
file string
}{
{
full: "",
dir: "",
file: "",
},
{
full: SelfDir,
dir: "",
file: SelfDir,
},
{
full: "rabbit.jpg",
dir: "",
file: "rabbit.jpg",
},
{
full: "/",
dir: "/",
file: "",
},
{
full: "/beans",
dir: "/",
file: "beans",
},
{
full: "/home/foo/bar",
dir: "/home/foo/",
file: "bar",
},
{
full: "/usr/local/",
dir: "/usr/local/",
file: "",
},
{
full: "/usr//local//go",
dir: "/usr//local//",
file: "go",
},
}
for _, p := range cases {
dir, file := filepath.Split(p.full)
if dir != p.dir || file != p.file {
t.Fatalf(
"in '%s',\ngot dir='%s' (expected '%s'),\n got file='%s' (expected %s).",
p.full, dir, p.dir, file, p.file)
}
}
}
func TestPathSplitAndJoin(t *testing.T) {
cases := map[string]struct {
original string
expected []string
}{
"Empty": {
original: "",
expected: []string{},
},
"One": {
original: "hello",
expected: []string{"hello"},
},
"Two": {
original: "hello/there",
expected: []string{"hello", "there"},
},
"Three": {
original: "hello/my/friend",
expected: []string{"hello", "my", "friend"},
},
}
for n, c := range cases {
f := func(t *testing.T, original string, expected []string) {
actual := PathSplit(original)
if len(actual) != len(expected) {
t.Fatalf(
"expected len %d, got len %d",
len(expected), len(actual))
}
for i := range expected {
if expected[i] != actual[i] {
t.Fatalf(
"at i=%d, expected '%s', got '%s'",
i, expected[i], actual[i])
}
}
joined := PathJoin(actual)
if joined != original {
t.Fatalf(
"when rejoining, expected '%s', got '%s'",
original, joined)
}
}
t.Run("relative"+n, func(t *testing.T) {
f(t, c.original, c.expected)
})
t.Run("absolute"+n, func(t *testing.T) {
f(t,
string(os.PathSeparator)+c.original,
append([]string{""}, c.expected...))
})
}
}
func TestInsertPathPart(t *testing.T) {
cases := map[string]struct {
original string
pos int
part string
expected string
}{
"rootOne": {
original: "/",
pos: 0,
part: "___",
expected: "/___",
},
"rootTwo": {
original: "/",
pos: 444,
part: "___",
expected: "/___",
},
"rootedFirst": {
original: "/apple",
pos: 0,
part: "___",
expected: "/___/apple",
},
"rootedSecond": {
original: "/apple",
pos: 444,
part: "___",
expected: "/apple/___",
},
"rootedThird": {
original: "/apple/banana",
pos: 444,
part: "___",
expected: "/apple/banana/___",
},
"emptyLow": {
original: "",
pos: -3,
part: "___",
expected: "___",
},
"emptyHigh": {
original: "",
pos: 444,
part: "___",
expected: "___",
},
"peachPie": {
original: "a/nice/warm/pie",
pos: 3,
part: "PEACH",
expected: "a/nice/warm/PEACH/pie",
},
"rootedPeachPie": {
original: "/a/nice/warm/pie",
pos: 3,
part: "PEACH",
expected: "/a/nice/warm/PEACH/pie",
},
"longStart": {
original: "a/b/c/d/e/f",
pos: 0,
part: "___",
expected: "___/a/b/c/d/e/f",
},
"rootedLongStart": {
original: "/a/b/c/d/e/f",
pos: 0,
part: "___",
expected: "/___/a/b/c/d/e/f",
},
"longMiddle": {
original: "a/b/c/d/e/f",
pos: 3,
part: "___",
expected: "a/b/c/___/d/e/f",
},
"rootedLongMiddle": {
original: "/a/b/c/d/e/f",
pos: 3,
part: "___",
expected: "/a/b/c/___/d/e/f",
},
"longEnd": {
original: "a/b/c/d/e/f",
pos: 444,
part: "___",
expected: "a/b/c/d/e/f/___",
},
"rootedLongEnd": {
original: "/a/b/c/d/e/f",
pos: 444,
part: "___",
expected: "/a/b/c/d/e/f/___",
},
}
for n, c := range cases {
t.Run(n, func(t *testing.T) {
actual := InsertPathPart(c.original, c.pos, c.part)
if actual != c.expected {
t.Fatalf("expected '%s', got '%s'", c.expected, actual)
}
})
}
}
func TestStripTrailingSeps(t *testing.T) {
cases := []struct {
full string
rem string
}{
{
full: "foo",
rem: "foo",
},
{
full: "",
rem: "",
},
{
full: "foo/",
rem: "foo",
},
{
full: "foo///bar///",
rem: "foo///bar",
},
{
full: "/////",
rem: "",
},
{
full: "/",
rem: "",
},
}
for _, p := range cases {
dir := StripTrailingSeps(p.full)
if dir != p.rem {
t.Fatalf(
"in '%s', got dir='%s' (expected '%s')",
p.full, dir, p.rem)
}
}
}
func TestStripLeadingSeps(t *testing.T) {
cases := []struct {
full string
rem string
}{
{
full: "foo",
rem: "foo",
},
{
full: "",
rem: "",
},
{
full: "/foo",
rem: "foo",
},
{
full: "///foo///bar///",
rem: "foo///bar///",
},
{
full: "/////",
rem: "",
},
{
full: "/",
rem: "",
},
}
for _, p := range cases {
dir := StripLeadingSeps(p.full)
if dir != p.rem {
t.Fatalf(
"in '%s', got dir='%s' (expected '%s')",
p.full, dir, p.rem)
}
}
}

View File

@@ -19,17 +19,9 @@ type Filter struct {
// FsSlice contains the FieldSpecs to locate the namespace field
FsSlice types.FsSlice
trackableSetter filtersutil.TrackableSetter
}
var _ kio.Filter = Filter{}
var _ kio.TrackableFilter = &Filter{}
// WithMutationTracker registers a callback which will be invoked each time a field is mutated
func (f *Filter) WithMutationTracker(callback func(key, value, tag string, node *yaml.RNode)) {
f.trackableSetter.WithMutationTracker(callback)
}
func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
keys := yaml.SortedMapKeys(f.Annotations)
@@ -38,7 +30,7 @@ func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
for _, k := range keys {
if err := node.PipeE(fsslice.Filter{
FsSlice: f.FsSlice,
SetValue: f.trackableSetter.SetEntry(
SetValue: filtersutil.SetEntry(
k, f.Annotations[k], yaml.NodeTagString),
CreateKind: yaml.MappingNode, // Annotations are MappingNodes.
CreateTag: yaml.NodeTagMap,

View File

@@ -11,20 +11,16 @@ import (
"sigs.k8s.io/kustomize/api/internal/plugins/builtinconfig"
filtertest_test "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
var annosFs = builtinconfig.MakeDefaultConfig().CommonAnnotations
func TestAnnotations_Filter(t *testing.T) {
mutationTrackStub := filtertest_test.MutationTrackerStub{}
testCases := map[string]struct {
input string
expectedOutput string
filter Filter
fsslice types.FsSlice
setEntryCallback func(key, value, tag string, node *yaml.RNode)
expectedSetEntryArgs []filtertest_test.SetValueArg
input string
expectedOutput string
filter Filter
fsslice types.FsSlice
}{
"add": {
input: `
@@ -214,86 +210,17 @@ metadata:
"b": "b1",
}},
},
// test usage of SetEntryCallback
"set_entry_callback": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
`,
expectedOutput: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
annotations:
a: a1
b: b1
spec:
template:
metadata:
annotations:
a: a1
b: b1
`,
filter: Filter{
Annotations: annoMap{
"a": "a1",
"b": "b1",
},
},
setEntryCallback: mutationTrackStub.MutationTracker,
fsslice: []types.FieldSpec{
{
Path: "spec/template/metadata/annotations",
CreateIfNotPresent: true,
},
},
expectedSetEntryArgs: []filtertest_test.SetValueArg{
{
Key: "a",
Value: "a1",
Tag: "!!str",
NodePath: []string{"metadata", "annotations"},
},
{
Key: "a",
Value: "a1",
Tag: "!!str",
NodePath: []string{"spec", "template", "metadata", "annotations"},
},
{
Key: "b",
Value: "b1",
Tag: "!!str",
NodePath: []string{"metadata", "annotations"},
},
{
Key: "b",
Value: "b1",
Tag: "!!str",
NodePath: []string{"spec", "template", "metadata", "annotations"},
},
},
},
}
for tn, tc := range testCases {
mutationTrackStub.Reset()
t.Run(tn, func(t *testing.T) {
filter := tc.filter
filter.WithMutationTracker(tc.setEntryCallback)
filter.FsSlice = append(annosFs, tc.fsslice...) //nolint:gocritic
filter.FsSlice = append(annosFs, tc.fsslice...)
if !assert.Equal(t,
strings.TrimSpace(tc.expectedOutput),
strings.TrimSpace(filtertest_test.RunFilter(t, tc.input, filter))) {
t.FailNow()
}
if !assert.Equal(t, tc.expectedSetEntryArgs, mutationTrackStub.SetValueArgs()) {
t.FailNow()
}
})
}
}

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package filters
// Package filters collects various implementations

View File

@@ -8,10 +8,10 @@ import (
"strings"
"sigs.k8s.io/kustomize/api/filters/filtersutil"
"sigs.k8s.io/kustomize/api/internal/utils"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/errors"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/kustomize/kyaml/utils"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
@@ -51,8 +51,9 @@ func (fltr Filter) Filter(obj *yaml.RNode) (*yaml.RNode, error) {
}
fltr.path = utils.PathSplitter(fltr.FieldSpec.Path, "/")
if err := fltr.filter(obj); err != nil {
s, _ := obj.String()
return nil, errors.WrapPrefixf(err,
"considering field '%s' of object %s", fltr.FieldSpec.Path, resid.FromRNode(obj))
"considering field '%s' of object\n%v", fltr.FieldSpec.Path, s)
}
return obj, nil
}
@@ -137,8 +138,6 @@ func (fltr Filter) handleMap(obj *yaml.RNode) error {
// seq calls filter on all sequence elements
func (fltr Filter) handleSequence(obj *yaml.RNode) error {
if err := obj.VisitElements(func(node *yaml.RNode) error {
// set an accurate FieldPath for nested elements
node.AppendToFieldPath(obj.FieldPath()...)
// recurse on each element -- re-allocating a Filter is
// not strictly required, but is more consistent with field
// and less likely to have side effects

View File

@@ -59,7 +59,11 @@ apiVersion: foo
kind: Bar
xxx:
`,
error: `considering field '' of object Bar.v1.foo/[noName].[noNs]: cannot set or create an empty field name`,
error: `considering field '' of object
apiVersion: foo/v1
kind: Bar
xxx:
: cannot set or create an empty field name`,
filter: fieldspec.Filter{
SetValue: filtersutil.SetScalar("e"),
},
@@ -212,7 +216,11 @@ kind: Bar
a:
b: a
`,
error: `considering field 'a/b/c' of object Bar.[noVer].[noGrp]/[noName].[noNs]: expected sequence or mapping node`,
error: `considering field 'a/b/c' of object
kind: Bar
a:
b: a
: expected sequence or mapping node`,
filter: fieldspec.Filter{
SetValue: filtersutil.SetScalar("e"),
},
@@ -558,85 +566,3 @@ a:
})
}
}
func TestFilter_FieldPaths(t *testing.T) {
testCases := map[string]struct {
input string
fieldSpec string
expected []string
}{
"fieldpath containing SequenceNode": {
input: `
apiVersion: v1
kind: Pod
metadata:
name: app
spec:
containers:
- name: store
image: redis:6.2.6
- name: server
image: nginx:latest
`,
fieldSpec: `
path: spec/containers[]/image
kind: Pod
`,
expected: []string{
"spec.containers.image",
"spec.containers.image",
},
},
"fieldpath with MappingNode": {
input: `
apiVersion: v1
kind: Pod
metadata:
name: app
spec:
containers:
- name: store
image: redis:6.2.6
- name: server
image: nginx:latest
`,
fieldSpec: `
path: metadata/name
kind: Pod
`,
expected: []string{
"metadata.name",
},
},
}
for name, tc := range testCases {
var fieldPaths []string
trackableSetter := filtersutil.TrackableSetter{}
trackableSetter.WithMutationTracker(func(key, value, tag string, node *yaml.RNode) {
fieldPaths = append(fieldPaths, strings.Join(node.FieldPath(), "."))
})
filter := fieldspec.Filter{
SetValue: trackableSetter.SetScalar("foo"),
}
t.Run(name, func(t *testing.T) {
err := yaml.Unmarshal([]byte(tc.fieldSpec), &filter.FieldSpec)
assert.NoError(t, err)
rw := &kio.ByteReadWriter{
Reader: bytes.NewBufferString(tc.input),
Writer: &bytes.Buffer{},
OmitReaderAnnotations: true,
}
// run the filter
err = kio.Pipeline{
Inputs: []kio.Reader{rw},
Filters: []kio.Filter{kio.FilterAll(filter)},
Outputs: []kio.Writer{rw},
}.Execute()
assert.NoError(t, err)
assert.Equal(t, tc.expected, fieldPaths)
})
}
}

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package filtersutil
import (
@@ -12,94 +9,25 @@ type SetFn func(*yaml.RNode) error
// SetScalar returns a SetFn to set a scalar value
func SetScalar(value string) SetFn {
return SetEntry("", value, yaml.NodeTagEmpty)
return func(node *yaml.RNode) error {
return node.PipeE(yaml.FieldSetter{StringValue: value})
}
}
// SetEntry returns a SetFn to set a field or a map entry to a value.
// It can be used with an empty name to set both a value and a tag on a scalar node.
// When setting only a value on a scalar node, use SetScalar instead.
func SetEntry(name, value, tag string) SetFn {
// SetEntry returns a SetFn to set an entry in a map
func SetEntry(key, value, tag string) SetFn {
n := &yaml.Node{
Kind: yaml.ScalarNode,
Value: value,
Tag: tag,
}
if tag == yaml.NodeTagString && yaml.IsYaml1_1NonString(n) {
n.Style = yaml.DoubleQuotedStyle
}
return func(node *yaml.RNode) error {
return node.PipeE(yaml.FieldSetter{
Name: name,
Name: key,
Value: yaml.NewRNode(n),
})
}
}
type TrackableSetter struct {
// SetValueCallback will be invoked each time a field is set
setValueCallback func(name, value, tag string, node *yaml.RNode)
}
// WithMutationTracker registers a callback which will be invoked each time a field is mutated
func (s *TrackableSetter) WithMutationTracker(callback func(key, value, tag string, node *yaml.RNode)) *TrackableSetter {
s.setValueCallback = callback
return s
}
// SetScalar returns a SetFn to set a scalar value.
// if a mutation tracker has been registered, the tracker will be invoked each
// time a scalar is set
func (s TrackableSetter) SetScalar(value string) SetFn {
return s.SetEntry("", value, yaml.NodeTagEmpty)
}
// SetScalarIfEmpty returns a SetFn to set a scalar value only if it isn't already set.
// If a mutation tracker has been registered, the tracker will be invoked each
// time a scalar is actually set.
func (s TrackableSetter) SetScalarIfEmpty(value string) SetFn {
return s.SetEntryIfEmpty("", value, yaml.NodeTagEmpty)
}
// SetEntry returns a SetFn to set a field or a map entry to a value.
// It can be used with an empty name to set both a value and a tag on a scalar node.
// When setting only a value on a scalar node, use SetScalar instead.
// If a mutation tracker has been registered, the tracker will be invoked each
// time an entry is set.
func (s TrackableSetter) SetEntry(name, value, tag string) SetFn {
origSetEntry := SetEntry(name, value, tag)
return func(node *yaml.RNode) error {
if s.setValueCallback != nil {
s.setValueCallback(name, value, tag, node)
}
return origSetEntry(node)
}
}
// SetEntryIfEmpty returns a SetFn to set a field or a map entry to a value only if it isn't already set.
// It can be used with an empty name to set both a value and a tag on a scalar node.
// When setting only a value on a scalar node, use SetScalar instead.
// If a mutation tracker has been registered, the tracker will be invoked each
// time an entry is actually set.
func (s TrackableSetter) SetEntryIfEmpty(key, value, tag string) SetFn {
origSetEntry := SetEntry(key, value, tag)
return func(node *yaml.RNode) error {
if hasExistingValue(node, key) {
return nil
}
if s.setValueCallback != nil {
s.setValueCallback(key, value, tag, node)
}
return origSetEntry(node)
}
}
func hasExistingValue(node *yaml.RNode, key string) bool {
if node.IsNilOrEmpty() {
return false
}
if err := yaml.ErrorIfInvalid(node, yaml.ScalarNode); err == nil {
return yaml.GetValue(node) != ""
}
entry := node.Field(key)
if entry.IsNilOrEmpty() {
return false
}
return yaml.GetValue(entry.Value) != ""
}

View File

@@ -1,108 +0,0 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package filtersutil_test
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sigs.k8s.io/kustomize/api/filters/filtersutil"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
func TestTrackableSetter_SetScalarIfEmpty(t *testing.T) {
tests := []struct {
name string
input *yaml.RNode
value string
want string
}{
{
name: "sets null values",
input: yaml.MakeNullNode(),
value: "foo",
want: "foo",
},
{
name: "sets empty values",
input: yaml.NewScalarRNode(""),
value: "foo",
want: "foo",
},
{
name: "does not overwrite values",
input: yaml.NewStringRNode("a"),
value: "foo",
want: "a",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
wasSet := false
s := (&filtersutil.TrackableSetter{}).WithMutationTracker(func(_, _, _ string, _ *yaml.RNode) {
wasSet = true
})
wantSet := tt.value == tt.want
fn := s.SetScalarIfEmpty(tt.value)
require.NoError(t, fn(tt.input))
assert.Equal(t, tt.want, yaml.GetValue(tt.input))
assert.Equal(t, wantSet, wasSet, "tracker invoked even though value was not changed")
})
}
}
func TestTrackableSetter_SetEntryIfEmpty(t *testing.T) {
tests := []struct {
name string
input *yaml.RNode
key string
value string
want string
}{
{
name: "sets empty values",
input: yaml.NewMapRNode(&map[string]string{"setMe": ""}),
key: "setMe",
value: "foo",
want: "foo",
},
{
name: "sets missing keys",
input: yaml.NewMapRNode(&map[string]string{}),
key: "setMe",
value: "foo",
want: "foo",
},
{
name: "does not overwrite values",
input: yaml.NewMapRNode(&map[string]string{"existing": "original"}),
key: "existing",
value: "foo",
want: "original",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
wasSet := false
s := (&filtersutil.TrackableSetter{}).WithMutationTracker(func(_, _, _ string, _ *yaml.RNode) {
wasSet = true
})
wantSet := tt.value == tt.want
fn := s.SetEntryIfEmpty(tt.key, tt.value, "")
require.NoError(t, fn(tt.input))
assert.Equal(t, tt.want, yaml.GetValue(tt.input.Field(tt.key).Value))
assert.Equal(t, wantSet, wasSet, "tracker invoked even though value was not changed")
})
}
}
func TestTrackableSetter_SetEntryIfEmpty_BadInputNodeKind(t *testing.T) {
fn := filtersutil.TrackableSetter{}.SetEntryIfEmpty("foo", "false", yaml.NodeTagBool)
rn := yaml.NewListRNode("nope")
rn.AppendToFieldPath("dummy", "path")
assert.EqualError(t, fn(rn), `wrong node kind: expected MappingNode but got SequenceNode: node contents:
- nope
`)
}

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package gkesagenerator contains a kio.Filter that that generates a
// iampolicy-related resources for a given cloud provider
package iampolicygenerator

View File

@@ -43,7 +43,7 @@ metadata:
f.IAMPolicyGenerator.KubernetesService.Name)
if f.IAMPolicyGenerator.Namespace != "" {
input += fmt.Sprintf("\n namespace: %s", f.IAMPolicyGenerator.Namespace)
input = input + fmt.Sprintf("\n namespace: %s", f.IAMPolicyGenerator.Namespace)
}
sa, err := yaml.Parse(input)

View File

@@ -23,17 +23,9 @@ type Filter struct {
// FsSlice contains the FieldSpecs to locate an image field,
// e.g. Path: "spec/myContainers[]/image"
FsSlice types.FsSlice `json:"fieldSpecs,omitempty" yaml:"fieldSpecs,omitempty"`
trackableSetter filtersutil.TrackableSetter
}
var _ kio.Filter = Filter{}
var _ kio.TrackableFilter = &Filter{}
// WithMutationTracker registers a callback which will be invoked each time a field is mutated
func (f *Filter) WithMutationTracker(callback func(key, value, tag string, node *yaml.RNode)) {
f.trackableSetter.WithMutationTracker(callback)
}
func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
_, err := kio.FilterAll(yaml.FilterFunc(f.filter)).Filter(nodes)
@@ -48,11 +40,8 @@ func (f Filter) filter(node *yaml.RNode) (*yaml.RNode, error) {
return node, nil
}
if err := node.PipeE(fsslice.Filter{
FsSlice: f.FsSlice,
SetValue: imageTagUpdater{
ImageTag: f.ImageTag,
trackableSetter: f.trackableSetter,
}.SetImageValue,
FsSlice: f.FsSlice,
SetValue: updateImageTagFn(f.ImageTag),
}); err != nil {
return nil, err
}
@@ -70,3 +59,11 @@ func (f Filter) isOnDenyList(node *yaml.RNode) bool {
// https://github.com/kubernetes-sigs/kustomize/issues/890
return meta.Kind == `CustomResourceDefinition`
}
func updateImageTagFn(imageTag types.Image) filtersutil.SetFn {
return func(node *yaml.RNode) error {
return node.PipeE(imageTagUpdater{
ImageTag: imageTag,
})
}
}

View File

@@ -10,18 +10,14 @@ import (
"github.com/stretchr/testify/assert"
filtertest "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
func TestImageTagUpdater_Filter(t *testing.T) {
mutationTrackerStub := filtertest.MutationTrackerStub{}
testCases := map[string]struct {
input string
expectedOutput string
filter Filter
fsSlice types.FsSlice
setValueCallback func(key, value, tag string, node *yaml.RNode)
expectedSetValueArgs []filtertest.SetValueArg
input string
expectedOutput string
filter Filter
fsSlice types.FsSlice
}{
"ignore CustomResourceDefinition": {
input: `
@@ -662,237 +658,17 @@ spec:
},
},
},
"mutation tracker": {
input: `
group: apps
apiVersion: v1
kind: Deployment
metadata:
name: deploy1
spec:
template:
spec:
containers:
- image: nginx:1.7.9
name: nginx-tagged
- image: nginx:latest
name: nginx-latest
- image: foobar:1
name: replaced-with-digest
- image: postgres:1.8.0
name: postgresdb
initContainers:
- image: nginx
name: nginx-notag
- image: nginx@sha256:111111111111111111
name: nginx-sha256
- image: alpine:1.8.0
name: init-alpine
`,
expectedOutput: `
group: apps
apiVersion: v1
kind: Deployment
metadata:
name: deploy1
spec:
template:
spec:
containers:
- image: busybox:v3
name: nginx-tagged
- image: busybox:v3
name: nginx-latest
- image: foobar:1
name: replaced-with-digest
- image: postgres:1.8.0
name: postgresdb
initContainers:
- image: busybox:v3
name: nginx-notag
- image: busybox:v3
name: nginx-sha256
- image: alpine:1.8.0
name: init-alpine
`,
filter: Filter{
ImageTag: types.Image{
Name: "nginx",
NewName: "busybox",
NewTag: "v3",
},
},
fsSlice: []types.FieldSpec{
{
Path: "spec/template/spec/containers[]/image",
},
{
Path: "spec/template/spec/initContainers[]/image",
},
},
setValueCallback: mutationTrackerStub.MutationTracker,
expectedSetValueArgs: []filtertest.SetValueArg{
{
Value: "busybox:v3",
NodePath: []string{"spec", "template", "spec", "containers", "image"},
},
{
Value: "busybox:v3",
NodePath: []string{"spec", "template", "spec", "containers", "image"},
},
{
Value: "busybox:v3",
NodePath: []string{"spec", "template", "spec", "initContainers", "image"},
},
{
Value: "busybox:v3",
NodePath: []string{"spec", "template", "spec", "initContainers", "image"},
},
},
},
"image with tag and digest new name": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
spec:
image: nginx:1.2.1@sha256:46d5b90a7f4e9996351ad893a26bcbd27216676ad4d5316088ce351fb2c2c3dd
`,
expectedOutput: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
spec:
image: apache:1.2.1@sha256:46d5b90a7f4e9996351ad893a26bcbd27216676ad4d5316088ce351fb2c2c3dd
`,
filter: Filter{
ImageTag: types.Image{
Name: "nginx",
NewName: "apache",
},
},
fsSlice: []types.FieldSpec{
{
Path: "spec/image",
},
},
},
"image with tag and digest new name new tag": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
spec:
image: nginx:1.2.1@sha256:46d5b90a7f4e9996351ad893a26bcbd27216676ad4d5316088ce351fb2c2c3dd
`,
expectedOutput: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
spec:
image: apache:1.3.0
`,
filter: Filter{
ImageTag: types.Image{
Name: "nginx",
NewName: "apache",
NewTag: "1.3.0",
},
},
fsSlice: []types.FieldSpec{
{
Path: "spec/image",
},
},
},
"image with tag and digest new name new tag and digest": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
spec:
image: nginx:1.2.1@sha256:46d5b90a7f4e9996351ad893a26bcbd27216676ad4d5316088ce351fb2c2c3dd
`,
expectedOutput: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
spec:
image: apache:1.3.0@sha256:xyz
`,
filter: Filter{
ImageTag: types.Image{
Name: "nginx",
NewName: "apache",
NewTag: "1.3.0",
Digest: "sha256:xyz",
},
},
fsSlice: []types.FieldSpec{
{
Path: "spec/image",
},
},
},
"updateimagesuffix": {
input: `
group: apps
apiVersion: v1
kind: Deployment
metadata:
name: deploysuffix
spec:
template:
spec:
containers:
- image: redis:6.2.6
name: redis
`,
expectedOutput: `
group: apps
apiVersion: v1
kind: Deployment
metadata:
name: deploysuffix
spec:
template:
spec:
containers:
- image: redis:6.2.6-alpine
name: redis
`,
filter: Filter{
ImageTag: types.Image{
Name: "redis",
TagSuffix: "-alpine",
},
},
fsSlice: []types.FieldSpec{
{
Path: "spec/template/spec/containers[]/image",
},
},
},
}
for tn, tc := range testCases {
mutationTrackerStub.Reset()
t.Run(tn, func(t *testing.T) {
filter := tc.filter
filter.WithMutationTracker(tc.setValueCallback)
filter.FsSlice = tc.fsSlice
if !assert.Equal(t,
strings.TrimSpace(tc.expectedOutput),
strings.TrimSpace(filtertest.RunFilter(t, tc.input, filter))) {
t.FailNow()
}
assert.Equal(t, tc.expectedSetValueArgs, mutationTrackerStub.SetValueArgs())
})
}
}

View File

@@ -4,9 +4,7 @@
package imagetag
import (
"sigs.k8s.io/kustomize/api/internal/utils"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/errors"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
@@ -76,17 +74,28 @@ func (f findFieldsFilter) walk(node *yaml.RNode) error {
return err
}
key := n.Key.YNode().Value
if utils.StringSliceContains(f.fields, key) {
if contains(f.fields, key) {
return f.fieldCallback(n.Value)
}
return nil
})
case yaml.SequenceNode:
return errors.Wrap(node.VisitElements(f.walk))
return node.VisitElements(func(n *yaml.RNode) error {
return f.walk(n)
})
}
return nil
}
func contains(slice []string, str string) bool {
for _, s := range slice {
if s == str {
return true
}
}
return false
}
func checkImageTagsFn(imageTag types.Image) fieldCallback {
return func(node *yaml.RNode) error {
if node.YNode().Kind != yaml.SequenceNode {

View File

@@ -4,8 +4,6 @@
package imagetag
import (
"sigs.k8s.io/kustomize/api/filters/filtersutil"
"sigs.k8s.io/kustomize/api/image"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/yaml"
@@ -15,57 +13,31 @@ import (
// that will update the value of the yaml node based on the provided
// ImageTag if the current value matches the format of an image reference.
type imageTagUpdater struct {
Kind string `yaml:"kind,omitempty"`
ImageTag types.Image `yaml:"imageTag,omitempty"`
trackableSetter filtersutil.TrackableSetter
Kind string `yaml:"kind,omitempty"`
ImageTag types.Image `yaml:"imageTag,omitempty"`
}
func (u imageTagUpdater) SetImageValue(rn *yaml.RNode) error {
func (u imageTagUpdater) Filter(rn *yaml.RNode) (*yaml.RNode, error) {
if err := yaml.ErrorIfInvalid(rn, yaml.ScalarNode); err != nil {
return err
return nil, err
}
value := rn.YNode().Value
if !image.IsImageMatched(value, u.ImageTag.Name) {
return nil
return rn, nil
}
name, tag, digest := image.Split(value)
name, tag := image.Split(value)
if u.ImageTag.NewName != "" {
name = u.ImageTag.NewName
}
// overriding tag or digest will replace both original tag and digest values
switch {
case u.ImageTag.NewTag != "" && u.ImageTag.Digest != "":
tag = u.ImageTag.NewTag
digest = u.ImageTag.Digest
case u.ImageTag.NewTag != "":
tag = u.ImageTag.NewTag
digest = ""
case u.ImageTag.Digest != "":
tag = ""
digest = u.ImageTag.Digest
case u.ImageTag.TagSuffix != "":
tag += u.ImageTag.TagSuffix
digest = ""
if u.ImageTag.NewTag != "" {
tag = ":" + u.ImageTag.NewTag
}
if u.ImageTag.Digest != "" {
tag = "@" + u.ImageTag.Digest
}
// build final image name
if tag != "" {
name += ":" + tag
}
if digest != "" {
name += "@" + digest
}
return u.trackableSetter.SetScalar(name)(rn)
}
func (u imageTagUpdater) Filter(rn *yaml.RNode) (*yaml.RNode, error) {
if err := u.SetImageValue(rn); err != nil {
return nil, err
}
return rn, nil
return rn.Pipe(yaml.FieldSetter{StringValue: name + tag})
}

View File

@@ -20,17 +20,9 @@ type Filter struct {
// FsSlice identifies the label fields.
FsSlice types.FsSlice
trackableSetter filtersutil.TrackableSetter
}
var _ kio.Filter = Filter{}
var _ kio.TrackableFilter = &Filter{}
// WithMutationTracker registers a callback which will be invoked each time a field is mutated
func (f *Filter) WithMutationTracker(callback func(key, value, tag string, node *yaml.RNode)) {
f.trackableSetter.WithMutationTracker(callback)
}
func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
keys := yaml.SortedMapKeys(f.Labels)
@@ -39,7 +31,7 @@ func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
for _, k := range keys {
if err := node.PipeE(fsslice.Filter{
FsSlice: f.FsSlice,
SetValue: f.trackableSetter.SetEntry(
SetValue: filtersutil.SetEntry(
k, f.Labels[k], yaml.NodeTagString),
CreateKind: yaml.MappingNode, // Labels are MappingNodes.
CreateTag: yaml.NodeTagMap,

View File

@@ -11,17 +11,13 @@ import (
filtertest_test "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
func TestLabels_Filter(t *testing.T) {
mutationTrackerStub := filtertest_test.MutationTrackerStub{}
testCases := map[string]struct {
input string
expectedOutput string
filter Filter
setEntryCallback func(key, value, tag string, node *yaml.RNode)
expectedSetEntryArgs []filtertest_test.SetValueArg
input string
expectedOutput string
filter Filter
}{
"add": {
input: `
@@ -403,74 +399,15 @@ metadata:
},
},
},
// test usage of SetEntryCallback
"set_entry_callback": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
labels:
witcher: geralt
`,
expectedOutput: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
labels:
witcher: geralt
mage: yennefer
a:
b:
mage: yennefer
`,
filter: Filter{
Labels: labelMap{
"mage": "yennefer",
},
FsSlice: []types.FieldSpec{
{
Path: "metadata/labels",
CreateIfNotPresent: true,
},
{
Path: "a/b",
CreateIfNotPresent: true,
},
},
},
setEntryCallback: mutationTrackerStub.MutationTracker,
expectedSetEntryArgs: []filtertest_test.SetValueArg{
{
Key: "mage",
Value: "yennefer",
Tag: "!!str",
NodePath: []string{"metadata", "labels"},
},
{
Key: "mage",
Value: "yennefer",
Tag: "!!str",
NodePath: []string{"a", "b"},
},
},
},
}
for tn, tc := range testCases {
mutationTrackerStub.Reset()
t.Run(tn, func(t *testing.T) {
tc.filter.WithMutationTracker(tc.setEntryCallback)
if !assert.Equal(t,
strings.TrimSpace(tc.expectedOutput),
strings.TrimSpace(filtertest_test.RunFilter(t, tc.input, tc.filter))) {
t.FailNow()
}
if !assert.Equal(t, tc.expectedSetEntryArgs, mutationTrackerStub.SetValueArgs()) {
t.FailNow()
}
})
}
}

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package nameref contains a kio.Filter implementation of the kustomize
// name reference transformer.
package nameref

View File

@@ -1,17 +1,14 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package nameref
import (
"fmt"
"strings"
"github.com/pkg/errors"
"sigs.k8s.io/kustomize/api/filters/fieldspec"
"sigs.k8s.io/kustomize/api/resmap"
"sigs.k8s.io/kustomize/api/resource"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/errors"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
@@ -59,12 +56,11 @@ func (f Filter) run(node *yaml.RNode) (*yaml.RNode, error) {
// sanity check.
return nil, err
}
f.NameFieldToUpdate.Gvk = f.Referrer.GetGvk()
if err := node.PipeE(fieldspec.Filter{
FieldSpec: f.NameFieldToUpdate,
SetValue: f.set,
}); err != nil {
return nil, errors.WrapPrefixf(
return nil, errors.Wrapf(
err, "updating name reference in '%s' field of '%s'",
f.NameFieldToUpdate.Path, f.Referrer.CurId().String())
}
@@ -104,7 +100,7 @@ func (f Filter) setMapping(node *yaml.RNode) error {
}
nameNode, err := node.Pipe(yaml.FieldMatcher{Name: "name"})
if err != nil {
return errors.WrapPrefixf(err, "trying to match 'name' field")
return errors.Wrap(err, "trying to match 'name' field")
}
if nameNode == nil {
// This is a _configuration_ error; the field path
@@ -118,9 +114,7 @@ func (f Filter) setMapping(node *yaml.RNode) error {
return err
}
oldName := nameNode.YNode().Value
// use allNamesAndNamespacesAreTheSame to compare referral candidates for functional identity,
// because we source both name and namespace values from the referral in this case.
referral, err := f.selectReferral(oldName, candidates, allNamesAndNamespacesAreTheSame)
referral, err := f.selectReferral(oldName, candidates)
if err != nil || referral == nil {
// Nil referral means nothing to do.
return err
@@ -153,7 +147,7 @@ func (f Filter) filterMapCandidatesByNamespace(
node *yaml.RNode) ([]*resource.Resource, error) {
namespaceNode, err := node.Pipe(yaml.FieldMatcher{Name: "namespace"})
if err != nil {
return nil, errors.WrapPrefixf(err, "trying to match 'namespace' field")
return nil, errors.Wrap(err, "trying to match 'namespace' field")
}
if namespaceNode == nil {
return f.ReferralCandidates.Resources(), nil
@@ -169,10 +163,8 @@ func (f Filter) filterMapCandidatesByNamespace(
}
func (f Filter) setScalar(node *yaml.RNode) error {
// use allNamesAreTheSame to compare referral candidates for functional identity,
// because we only source the name from the referral in this case.
referral, err := f.selectReferral(
node.YNode().Value, f.ReferralCandidates.Resources(), allNamesAreTheSame)
node.YNode().Value, f.ReferralCandidates.Resources())
if err != nil || referral == nil {
// Nil referral means nothing to do.
return err
@@ -315,9 +307,7 @@ func (f Filter) sameCurrentNamespaceAsReferrer() sieveFunc {
func (f Filter) selectReferral(
// The name referral that may need to be updated.
oldName string,
candidates []*resource.Resource,
// function that returns whether two referrals are identical for the purposes of the transformation
candidatesIdentical func(resources []*resource.Resource) bool) (*resource.Resource, error) {
candidates []*resource.Resource) (*resource.Resource, error) {
candidates = doSieve(candidates, previousNameMatches(oldName))
candidates = doSieve(candidates, previousIdSelectedByGvk(&f.ReferralTarget))
candidates = doSieve(candidates, f.roleRefFilter())
@@ -332,21 +322,24 @@ func (f Filter) selectReferral(
if len(candidates) == 0 {
return nil, nil
}
if candidatesIdentical(candidates) {
if allNamesAreTheSame(candidates) {
// Just take the first one.
return candidates[0], nil
}
ids := getIds(candidates)
return nil, fmt.Errorf("found multiple possible referrals: %s\n%s", ids, f.failureDetails(candidates))
f.failureDetails(candidates)
return nil, fmt.Errorf(" found multiple possible referrals: %s", ids)
}
func (f Filter) failureDetails(resources []*resource.Resource) string {
msg := strings.Builder{}
msg.WriteString(fmt.Sprintf("\n**** Too many possible referral targets to referrer:\n%s\n", f.Referrer.MustYaml()))
func (f Filter) failureDetails(resources []*resource.Resource) {
fmt.Printf(
"\n**** Too many possible referral targets to referrer:\n%s\n",
f.Referrer.MustYaml())
for i, r := range resources {
msg.WriteString(fmt.Sprintf("--- possible referral %d:\n%s\n", i, r.MustYaml()))
fmt.Printf(
"--- possible referral %d:\n%s", i, r.MustYaml())
fmt.Println("------")
}
return msg.String()
}
func allNamesAreTheSame(resources []*resource.Resource) bool {
@@ -359,17 +352,6 @@ func allNamesAreTheSame(resources []*resource.Resource) bool {
return true
}
func allNamesAndNamespacesAreTheSame(resources []*resource.Resource) bool {
name := resources[0].GetName()
namespace := resources[0].GetNamespace()
for i := 1; i < len(resources); i++ {
if name != resources[i].GetName() || namespace != resources[i].GetNamespace() {
return false
}
}
return true
}
func getIds(rs []*resource.Resource) string {
var result []string
for _, r := range rs {

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package nameref
import (

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package nameref
import (

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package nameref
import (

View File

@@ -7,7 +7,6 @@ import (
"sigs.k8s.io/kustomize/api/filters/filtersutil"
"sigs.k8s.io/kustomize/api/filters/fsslice"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/errors"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/resid"
"sigs.k8s.io/kustomize/kyaml/yaml"
@@ -19,36 +18,9 @@ type Filter struct {
// FsSlice contains the FieldSpecs to locate the namespace field
FsSlice types.FsSlice `json:"fieldSpecs,omitempty" yaml:"fieldSpecs,omitempty"`
// UnsetOnly means only blank namespace fields will be set
UnsetOnly bool `json:"unsetOnly" yaml:"unsetOnly"`
// SetRoleBindingSubjects determines which subject fields in RoleBinding and ClusterRoleBinding
// objects will have their namespace fields set. Overrides field specs provided for these types, if any.
// - defaultOnly (default): namespace will be set only on subjects named "default".
// - allServiceAccounts: namespace will be set on all subjects with "kind: ServiceAccount"
// - none: all subjects will be skipped.
SetRoleBindingSubjects RoleBindingSubjectMode `json:"setRoleBindingSubjects" yaml:"setRoleBindingSubjects"`
trackableSetter filtersutil.TrackableSetter
}
type RoleBindingSubjectMode string
const (
DefaultSubjectsOnly RoleBindingSubjectMode = "defaultOnly"
SubjectModeUnspecified RoleBindingSubjectMode = ""
AllServiceAccountSubjects RoleBindingSubjectMode = "allServiceAccounts"
NoSubjects RoleBindingSubjectMode = "none"
)
var _ kio.Filter = Filter{}
var _ kio.TrackableFilter = &Filter{}
// WithMutationTracker registers a callback which will be invoked each time a field is mutated
func (ns *Filter) WithMutationTracker(callback func(key, value, tag string, node *yaml.RNode)) {
ns.trackableSetter.WithMutationTracker(callback)
}
func (ns Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
return kio.FilterAll(yaml.FilterFunc(ns.run)).Filter(nodes)
@@ -56,40 +28,47 @@ func (ns Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
// Run runs the filter on a single node rather than a slice
func (ns Filter) run(node *yaml.RNode) (*yaml.RNode, error) {
// Special handling for metadata.namespace and metadata.name -- :(
// never let SetEntry handle metadata.namespace--it will incorrectly include cluster-scoped resources
// only update metadata.name if api version is expected one--so-as it leaves other resources of kind namespace alone
apiVersion := node.GetApiVersion()
ns.FsSlice = ns.removeUnneededMetaFieldSpecs(apiVersion, ns.FsSlice)
gvk := resid.GvkFromNode(node)
if err := ns.metaNamespaceHack(node, gvk); err != nil {
// hacks for hardcoded types -- :(
if err := ns.hacks(node); err != nil {
return nil, err
}
// Special handling for (cluster) role binding subjects -- :(
if isRoleBinding(gvk.Kind) {
ns.FsSlice = ns.removeRoleBindingSubjectFieldSpecs(ns.FsSlice)
if err := ns.roleBindingHack(node); err != nil {
return nil, err
}
}
// Remove the fieldspecs that are for hardcoded fields. The fieldspecs
// exist for backwards compatibility with other implementations
// of this transformation.
// This implementation of the namespace transformation
// Does not use the fieldspecs for implementing cases which
// require hardcoded logic.
ns.FsSlice = ns.removeFieldSpecsForHacks(ns.FsSlice)
// transformations based on data -- :)
err := node.PipeE(fsslice.Filter{
FsSlice: ns.FsSlice,
SetValue: ns.fieldSetter(),
SetValue: filtersutil.SetScalar(ns.Namespace),
CreateKind: yaml.ScalarNode, // Namespace is a ScalarNode
CreateTag: yaml.NodeTagString,
})
invalidKindErr := &yaml.InvalidNodeKindError{}
if err != nil && errors.As(err, &invalidKindErr) && invalidKindErr.ActualNodeKind() != yaml.ScalarNode {
return nil, errors.WrapPrefixf(err, "namespace field specs must target scalar nodes")
return node, err
}
// hacks applies the namespace transforms that are hardcoded rather
// than specified through FieldSpecs.
func (ns Filter) hacks(obj *yaml.RNode) error {
gvk := resid.GvkFromNode(obj)
if err := ns.metaNamespaceHack(obj, gvk); err != nil {
return err
}
return node, errors.WrapPrefixf(err, "namespace transformation failed")
return ns.roleBindingHack(obj, gvk)
}
// metaNamespaceHack is a hack for implementing the namespace transform
// for the metadata.namespace field on namespace scoped resources.
// namespace scoped resources are determined by NOT being present
// in a hard-coded list of cluster-scoped resource types (by apiVersion and kind).
//
// This hack should be updated to allow individual resources to specify
// if they are cluster scoped through either an annotation on the resources,
// or through inlined OpenAPI on the resource as a YAML comment.
func (ns Filter) metaNamespaceHack(obj *yaml.RNode, gvk resid.Gvk) error {
if gvk.IsClusterScoped() {
return nil
@@ -98,23 +77,19 @@ func (ns Filter) metaNamespaceHack(obj *yaml.RNode, gvk resid.Gvk) error {
FsSlice: []types.FieldSpec{
{Path: types.MetadataNamespacePath, CreateIfNotPresent: true},
},
SetValue: ns.fieldSetter(),
SetValue: filtersutil.SetScalar(ns.Namespace),
CreateKind: yaml.ScalarNode, // Namespace is a ScalarNode
}
_, err := f.Filter(obj)
return err
}
// roleBindingHack is a hack for implementing the transformer's SetRoleBindingSubjects option
// roleBindingHack is a hack for implementing the namespace transform
// for RoleBinding and ClusterRoleBinding resource types.
//
// In NoSubjects mode, it does nothing.
//
// In AllServiceAccountSubjects mode, it sets the namespace on subjects with "kind: ServiceAccount".
//
// In DefaultSubjectsOnly mode (default mode), RoleBinding and ClusterRoleBinding have namespace set on
// RoleBinding and ClusterRoleBinding have namespace set on
// elements of the "subjects" field if and only if the subject elements
// "name" is "default". Otherwise the namespace is not set.
//
// Example:
//
// kind: RoleBinding
@@ -123,78 +98,56 @@ func (ns Filter) metaNamespaceHack(obj *yaml.RNode, gvk resid.Gvk) error {
// ...
// - name: "something-else" # this will not have the namespace set
// ...
func (ns Filter) roleBindingHack(obj *yaml.RNode) error {
var visitor filtersutil.SetFn
switch ns.SetRoleBindingSubjects {
case NoSubjects:
func (ns Filter) roleBindingHack(obj *yaml.RNode, gvk resid.Gvk) error {
if gvk.Kind != roleBindingKind && gvk.Kind != clusterRoleBindingKind {
return nil
case DefaultSubjectsOnly, SubjectModeUnspecified:
visitor = ns.setSubjectsNamedDefault
case AllServiceAccountSubjects:
visitor = ns.setServiceAccountNamespaces
default:
return errors.Errorf("invalid value %q for setRoleBindingSubjects: "+
"must be one of %q, %q or %q", ns.SetRoleBindingSubjects,
DefaultSubjectsOnly, NoSubjects, AllServiceAccountSubjects)
}
// Lookup the subjects field on all elements.
// Lookup the namespace field on all elements.
// We should change the fieldspec so this isn't necessary.
obj, err := obj.Pipe(yaml.Lookup(subjectsField))
if err != nil || yaml.IsMissingOrNull(obj) {
return err
}
// Use the appropriate visitor to set the namespace field on the correct subset of subjects
return errors.WrapPrefixf(obj.VisitElements(visitor), "setting namespace on (cluster)role binding subjects")
}
func isRoleBinding(kind string) bool {
return kind == roleBindingKind || kind == clusterRoleBindingKind
}
func (ns Filter) setServiceAccountNamespaces(o *yaml.RNode) error {
name, err := o.Pipe(yaml.Lookup("kind"), yaml.Match("ServiceAccount"))
if err != nil || yaml.IsMissingOrNull(name) {
return errors.WrapPrefixf(err, "looking up kind on (cluster)role binding subject")
}
return setNamespaceField(o, ns.fieldSetter())
}
func (ns Filter) setSubjectsNamedDefault(o *yaml.RNode) error {
name, err := o.Pipe(yaml.Lookup("name"), yaml.Match("default"))
if err != nil || yaml.IsMissingOrNull(name) {
return errors.WrapPrefixf(err, "looking up name on (cluster)role binding subject")
}
return setNamespaceField(o, ns.fieldSetter())
}
func setNamespaceField(node *yaml.RNode, setter filtersutil.SetFn) error {
node, err := node.Pipe(yaml.LookupCreate(yaml.ScalarNode, "namespace"))
if err != nil {
return errors.WrapPrefixf(err, "setting namespace field on (cluster)role binding subject")
}
return setter(node)
}
// removeRoleBindingSubjectFieldSpecs removes from the list fieldspecs that
// have hardcoded implementations
func (ns Filter) removeRoleBindingSubjectFieldSpecs(fs types.FsSlice) types.FsSlice {
var val types.FsSlice
for i := range fs {
if isRoleBinding(fs[i].Kind) && fs[i].Path == subjectsNamespacePath {
continue
// add the namespace to each "subject" with name: default
err = obj.VisitElements(func(o *yaml.RNode) error {
// The only case we need to force the namespace
// if for the "service account". "default" is
// kind of hardcoded here for right now.
name, err := o.Pipe(
yaml.Lookup("name"), yaml.Match("default"),
)
if err != nil || yaml.IsMissingOrNull(name) {
return err
}
val = append(val, fs[i])
}
return val
// set the namespace for the default account
v := yaml.NewScalarRNode(ns.Namespace)
return o.PipeE(
yaml.LookupCreate(yaml.ScalarNode, "namespace"),
yaml.FieldSetter{Value: v},
)
})
return err
}
func (ns Filter) removeUnneededMetaFieldSpecs(apiVersion string, fs types.FsSlice) types.FsSlice {
// removeFieldSpecsForHacks removes from the list fieldspecs that
// have hardcoded implementations
func (ns Filter) removeFieldSpecsForHacks(fs types.FsSlice) types.FsSlice {
var val types.FsSlice
for i := range fs {
// implemented by metaNamespaceHack
if fs[i].Path == types.MetadataNamespacePath {
continue
}
if apiVersion != types.MetadataNamespaceApiVersion && fs[i].Path == types.MetadataNamePath {
// implemented by roleBindingHack
if fs[i].Kind == roleBindingKind && fs[i].Path == subjectsField {
continue
}
// implemented by roleBindingHack
if fs[i].Kind == clusterRoleBindingKind && fs[i].Path == subjectsField {
continue
}
val = append(val, fs[i])
@@ -202,16 +155,8 @@ func (ns Filter) removeUnneededMetaFieldSpecs(apiVersion string, fs types.FsSlic
return val
}
func (ns *Filter) fieldSetter() filtersutil.SetFn {
if ns.UnsetOnly {
return ns.trackableSetter.SetEntryIfEmpty("", ns.Namespace, yaml.NodeTagString)
}
return ns.trackableSetter.SetEntry("", ns.Namespace, yaml.NodeTagString)
}
const (
subjectsField = "subjects"
subjectsNamespacePath = "subjects/namespace"
roleBindingKind = "RoleBinding"
clusterRoleBindingKind = "ClusterRoleBinding"
)

View File

@@ -12,11 +12,8 @@ import (
"sigs.k8s.io/kustomize/api/internal/plugins/builtinconfig"
filtertest_test "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
var mutationTrackerStub = filtertest_test.MutationTrackerStub{}
var tests = []TestCase{
{
name: "add",
@@ -286,134 +283,29 @@ a:
},
},
},
{
name: "mutation tracker",
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: RoleBinding
subjects:
- name: default
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
namespace: bar
a:
b:
c: bar
---
apiVersion: example.com/v1
kind: RoleBinding
subjects:
- name: default
namespace: bar
metadata:
namespace: bar
a:
b:
c: bar
`,
filter: namespace.Filter{Namespace: "bar"},
fsslice: []types.FieldSpec{
{
Path: "a/b/c",
CreateIfNotPresent: true,
},
},
mutationTracker: mutationTrackerStub.MutationTracker,
expectedSetValueArgs: []filtertest_test.SetValueArg{
{
Value: "bar",
Tag: "!!str",
NodePath: []string{"metadata", "namespace"},
},
{
Value: "bar",
Tag: "!!str",
NodePath: []string{"a", "b", "c"},
},
{
Value: "bar",
Tag: "!!str",
NodePath: []string{"metadata", "namespace"},
},
{
Value: "bar",
Tag: "!!str",
NodePath: []string{"namespace"},
},
{
Value: "bar",
Tag: "!!str",
NodePath: []string{"a", "b", "c"},
},
},
},
{
name: "numeric",
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
namespace: "01234"
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
namespace: "01234"
`,
filter: namespace.Filter{Namespace: "01234"},
},
}
type TestCase struct {
name string
input string
expected string
filter namespace.Filter
fsslice types.FsSlice
mutationTracker func(key, value, tag string, node *yaml.RNode)
expectedSetValueArgs []filtertest_test.SetValueArg
name string
input string
expected string
filter namespace.Filter
fsslice types.FsSlice
}
var config = builtinconfig.MakeDefaultConfig()
func TestNamespace_Filter(t *testing.T) {
for i := range tests {
mutationTrackerStub.Reset()
test := tests[i]
test.filter.WithMutationTracker(test.mutationTracker)
t.Run(test.name, func(t *testing.T) {
test.filter.FsSlice = append(config.NameSpace, test.fsslice...) //nolint:gocritic
test.filter.FsSlice = append(config.NameSpace, test.fsslice...)
if !assert.Equal(t,
strings.TrimSpace(test.expected),
strings.TrimSpace(
filtertest_test.RunFilter(t, test.input, test.filter))) {
t.FailNow()
}
assert.Equal(t, test.expectedSetValueArgs, mutationTrackerStub.SetValueArgs())
})
}
}

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package patchstrategicmerge
import (
@@ -732,50 +729,6 @@ spec:
protocol: "TCP"
- containerPort: 8301
protocol: "UDP"
`,
},
// Issue #4628
"should retain existing null values in targets": {
input: `
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: chart
spec:
releaseName: helm-chart
timeout: 15m
values:
chart:
replicaCount: null
autoscaling: true
`,
patch: yaml.MustParse(`
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: chart
spec:
releaseName: helm-chart
timeout: 15m
values:
deepgram-api:
some: value
`),
expected: `
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: chart
spec:
releaseName: helm-chart
timeout: 15m
values:
chart:
replicaCount: null
autoscaling: true
deepgram-api:
some: value
`,
},
}

View File

@@ -1,6 +0,0 @@
// Copyright 2020 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package prefix contains a kio.Filter implementation of the kustomize
// PrefixTransformer.
package prefix

View File

@@ -1,47 +0,0 @@
// Copyright 2020 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package prefix_test
import (
"bytes"
"log"
"os"
"sigs.k8s.io/kustomize/api/filters/prefix"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
)
func ExampleFilter() {
err := kio.Pipeline{
Inputs: []kio.Reader{&kio.ByteReader{Reader: bytes.NewBufferString(`
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`)}},
Filters: []kio.Filter{prefix.Filter{
Prefix: "baz-", FieldSpec: types.FieldSpec{Path: "metadata/name"}}},
Outputs: []kio.Writer{kio.ByteWriter{Writer: os.Stdout}},
}.Execute()
if err != nil {
log.Fatal(err)
}
// Output:
// apiVersion: example.com/v1
// kind: Foo
// metadata:
// name: baz-instance
// ---
// apiVersion: example.com/v1
// kind: Bar
// metadata:
// name: baz-instance
}

View File

@@ -1,50 +0,0 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package prefix
import (
"fmt"
"sigs.k8s.io/kustomize/api/filters/fieldspec"
"sigs.k8s.io/kustomize/api/filters/filtersutil"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
// Filter applies resource name prefix's using the fieldSpecs
type Filter struct {
Prefix string `json:"prefix,omitempty" yaml:"prefix,omitempty"`
FieldSpec types.FieldSpec `json:"fieldSpec,omitempty" yaml:"fieldSpec,omitempty"`
trackableSetter filtersutil.TrackableSetter
}
var _ kio.Filter = Filter{}
var _ kio.TrackableFilter = &Filter{}
// WithMutationTracker registers a callback which will be invoked each time a field is mutated
func (f *Filter) WithMutationTracker(callback func(key, value, tag string, node *yaml.RNode)) {
f.trackableSetter.WithMutationTracker(callback)
}
func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
return kio.FilterAll(yaml.FilterFunc(f.run)).Filter(nodes)
}
func (f Filter) run(node *yaml.RNode) (*yaml.RNode, error) {
err := node.PipeE(fieldspec.Filter{
FieldSpec: f.FieldSpec,
SetValue: f.evaluateField,
CreateKind: yaml.ScalarNode, // Name is a ScalarNode
CreateTag: yaml.NodeTagString,
})
return node, err
}
func (f Filter) evaluateField(node *yaml.RNode) error {
return f.trackableSetter.SetScalar(fmt.Sprintf(
"%s%s", f.Prefix, node.YNode().Value))(node)
}

View File

@@ -1,154 +0,0 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package prefix_test
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"sigs.k8s.io/kustomize/api/filters/prefix"
filtertest_test "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
var mutationTrackerStub = filtertest_test.MutationTrackerStub{}
var tests = map[string]TestCase{
"prefix": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: foo-instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: foo-instance
`,
filter: prefix.Filter{
Prefix: "foo-",
FieldSpec: types.FieldSpec{Path: "metadata/name"},
},
},
"data-fieldspecs": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
a:
b:
c: d
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
a:
b:
c: d
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
a:
b:
c: foo-d
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
a:
b:
c: foo-d
`,
filter: prefix.Filter{
Prefix: "foo-",
FieldSpec: types.FieldSpec{Path: "a/b/c"},
},
},
"mutation tracker": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: foo-instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: foo-instance
`,
filter: prefix.Filter{
Prefix: "foo-",
FieldSpec: types.FieldSpec{Path: "metadata/name"},
},
mutationTracker: mutationTrackerStub.MutationTracker,
expectedSetValueArgs: []filtertest_test.SetValueArg{
{
Value: "foo-instance",
NodePath: []string{"metadata", "name"},
},
{
Value: "foo-instance",
NodePath: []string{"metadata", "name"},
},
},
},
}
type TestCase struct {
input string
expected string
filter prefix.Filter
mutationTracker func(key, value, tag string, node *yaml.RNode)
expectedSetValueArgs []filtertest_test.SetValueArg
}
func TestFilter(t *testing.T) {
for name := range tests {
mutationTrackerStub.Reset()
test := tests[name]
test.filter.WithMutationTracker(test.mutationTracker)
t.Run(name, func(t *testing.T) {
if !assert.Equal(t,
strings.TrimSpace(test.expected),
strings.TrimSpace(
filtertest_test.RunFilter(t, test.input, test.filter))) {
t.FailNow()
}
assert.Equal(t, test.expectedSetValueArgs, mutationTrackerStub.SetValueArgs())
})
}
}

View File

@@ -0,0 +1,6 @@
// Copyright 2020 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package prefixsuffix contains a kio.Filter implementation of the kustomize
// PrefixSuffixTransformer.
package prefixsuffix

View File

@@ -0,0 +1,47 @@
// Copyright 2020 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package prefixsuffix_test
import (
"bytes"
"log"
"os"
"sigs.k8s.io/kustomize/api/filters/prefixsuffix"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
)
func ExampleFilter() {
err := kio.Pipeline{
Inputs: []kio.Reader{&kio.ByteReader{Reader: bytes.NewBufferString(`
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`)}},
Filters: []kio.Filter{prefixsuffix.Filter{
Prefix: "baz-", FieldSpec: types.FieldSpec{Path: "metadata/name"}}},
Outputs: []kio.Writer{kio.ByteWriter{Writer: os.Stdout}},
}.Execute()
if err != nil {
log.Fatal(err)
}
// Output:
// apiVersion: example.com/v1
// kind: Foo
// metadata:
// name: baz-instance
// ---
// apiVersion: example.com/v1
// kind: Bar
// metadata:
// name: baz-instance
}

View File

@@ -0,0 +1,43 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package prefixsuffix
import (
"fmt"
"sigs.k8s.io/kustomize/api/filters/fieldspec"
"sigs.k8s.io/kustomize/api/filters/filtersutil"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
// Filter applies resource name prefix's and suffix's using the fieldSpecs
type Filter struct {
Prefix string `json:"prefix,omitempty" yaml:"prefix,omitempty"`
Suffix string `json:"suffix,omitempty" yaml:"suffix,omitempty"`
FieldSpec types.FieldSpec `json:"fieldSpec,omitempty" yaml:"fieldSpec,omitempty"`
}
var _ kio.Filter = Filter{}
func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
return kio.FilterAll(yaml.FilterFunc(f.run)).Filter(nodes)
}
func (f Filter) run(node *yaml.RNode) (*yaml.RNode, error) {
err := node.PipeE(fieldspec.Filter{
FieldSpec: f.FieldSpec,
SetValue: f.evaluateField,
CreateKind: yaml.ScalarNode, // Name is a ScalarNode
CreateTag: yaml.NodeTagString,
})
return node, err
}
func (f Filter) evaluateField(node *yaml.RNode) error {
return filtersutil.SetScalar(fmt.Sprintf(
"%s%s%s", f.Prefix, node.YNode().Value, f.Suffix))(node)
}

View File

@@ -0,0 +1,158 @@
// Copyright 2019 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package prefixsuffix_test
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"sigs.k8s.io/kustomize/api/filters/prefixsuffix"
filtertest_test "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
)
var tests = map[string]TestCase{
"prefix": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: foo-instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: foo-instance
`,
filter: prefixsuffix.Filter{Prefix: "foo-"},
fs: types.FieldSpec{Path: "metadata/name"},
},
"suffix": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance-foo
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance-foo
`,
filter: prefixsuffix.Filter{Suffix: "-foo"},
fs: types.FieldSpec{Path: "metadata/name"},
},
"prefix-suffix": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: bar-instance-foo
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: bar-instance-foo
`,
filter: prefixsuffix.Filter{Prefix: "bar-", Suffix: "-foo"},
fs: types.FieldSpec{Path: "metadata/name"},
},
"data-fieldspecs": {
input: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
a:
b:
c: d
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
a:
b:
c: d
`,
expected: `
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
a:
b:
c: foo-d
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
a:
b:
c: foo-d
`,
filter: prefixsuffix.Filter{Prefix: "foo-"},
fs: types.FieldSpec{Path: "a/b/c"},
},
}
type TestCase struct {
input string
expected string
filter prefixsuffix.Filter
fs types.FieldSpec
}
func TestFilter(t *testing.T) {
for name := range tests {
test := tests[name]
t.Run(name, func(t *testing.T) {
test.filter.FieldSpec = test.fs
if !assert.Equal(t,
strings.TrimSpace(test.expected),
strings.TrimSpace(
filtertest_test.RunFilter(t, test.input, test.filter))) {
t.FailNow()
}
})
}
}

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package refvar contains a kio.Filter implementation of the kustomize
// refvar transformer (find and replace $(FOO) style variables in strings).
package refvar

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package refvar
import (

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package refvar_test
import (
@@ -20,6 +17,7 @@ var makeMf = func(theMap map[string]interface{}) MappingFunc {
}
func TestFilter(t *testing.T) {
testCases := map[string]struct {
input string
expected string
@@ -245,7 +243,17 @@ metadata:
data:
slice:
- false`,
expectedError: `considering field 'data/slice' of object Deployment.v1.apps/dep.[noNs]: invalid value type expect a string`,
expectedError: `considering field 'data/slice' of object
apiVersion: apps/v1
kind: Deployment
metadata:
name: dep
annotations:
config.kubernetes.io/index: '0'
data:
slice:
- false
: invalid value type expect a string`,
filter: Filter{
MappingFunc: makeMf(map[string]interface{}{
"VAR": int64(5),
@@ -261,7 +269,16 @@ metadata:
name: dep
data:
1: str`,
expectedError: `considering field 'data' of object Deployment.v1.apps/dep.[noNs]: invalid map key: value='1', tag='` + yaml.NodeTagInt + `'`,
expectedError: `considering field 'data' of object
apiVersion: apps/v1
kind: Deployment
metadata:
name: dep
annotations:
config.kubernetes.io/index: '0'
data:
1: str
: invalid map key: value='1', tag='` + yaml.NodeTagInt + `'`,
filter: Filter{
MappingFunc: makeMf(map[string]interface{}{
"VAR": int64(5),

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package replacement contains a kio.Filter implementation of the kustomize
// replacement transformer (accepts sources and looks for targets to replace
// their values with values from the sources).

View File

@@ -8,11 +8,8 @@ import (
"strings"
"sigs.k8s.io/kustomize/api/internal/utils"
"sigs.k8s.io/kustomize/api/resource"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/errors"
"sigs.k8s.io/kustomize/kyaml/resid"
kyaml_utils "sigs.k8s.io/kustomize/kyaml/utils"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
@@ -22,11 +19,11 @@ type Filter struct {
// Filter replaces values of targets with values from sources
func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
for i, r := range f.Replacements {
for _, r := range f.Replacements {
if r.Source == nil || r.Targets == nil {
return nil, fmt.Errorf("replacements must specify a source and at least one target")
}
value, err := getReplacement(nodes, &f.Replacements[i])
value, err := getReplacement(nodes, &r)
if err != nil {
return nil, err
}
@@ -38,6 +35,81 @@ func (f Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
return nodes, nil
}
func applyReplacement(nodes []*yaml.RNode, value *yaml.RNode, targets []*types.TargetSelector) ([]*yaml.RNode, error) {
for _, t := range targets {
if t.Select == nil {
return nil, fmt.Errorf("target must specify resources to select")
}
if len(t.FieldPaths) == 0 {
t.FieldPaths = []string{types.DefaultReplacementFieldPath}
}
for _, n := range nodes {
id := makeResId(n)
if id.IsSelectedBy(t.Select.ResId) && !rejectId(t.Reject, id) {
err := applyToNode(n, value, t)
if err != nil {
return nil, err
}
}
}
}
return nodes, nil
}
func rejectId(rejects []*types.Selector, id *resid.ResId) bool {
for _, r := range rejects {
if id.IsSelectedBy(r.ResId) {
return true
}
}
return false
}
func applyToNode(node *yaml.RNode, value *yaml.RNode, target *types.TargetSelector) error {
for _, fp := range target.FieldPaths {
fieldPath := utils.SmarterPathSplitter(fp, ".")
var t *yaml.RNode
var err error
if target.Options != nil && target.Options.Create {
t, err = node.Pipe(yaml.LookupCreate(value.YNode().Kind, fieldPath...))
} else {
t, err = node.Pipe(yaml.Lookup(fieldPath...))
}
if err != nil {
return err
}
if t != nil {
if err = setTargetValue(target.Options, t, value); err != nil {
return err
}
}
}
return nil
}
func setTargetValue(options *types.FieldOptions, t *yaml.RNode, value *yaml.RNode) error {
value = value.Copy()
if options != nil && options.Delimiter != "" {
if t.YNode().Kind != yaml.ScalarNode {
return fmt.Errorf("delimiter option can only be used with scalar nodes")
}
tv := strings.Split(t.YNode().Value, options.Delimiter)
v := yaml.GetValue(value)
// TODO: Add a way to remove an element
switch {
case options.Index < 0: // prefix
tv = append([]string{v}, tv...)
case options.Index >= len(tv): // suffix
tv = append(tv, v)
default: // replace an element
tv[options.Index] = v
}
value.YNode().Value = strings.Join(tv, options.Delimiter)
}
t.SetYNode(value.YNode())
return nil
}
func getReplacement(nodes []*yaml.RNode, r *types.Replacement) (*yaml.RNode, error) {
source, err := selectSourceNode(nodes, r.Source)
if err != nil {
@@ -47,43 +119,16 @@ func getReplacement(nodes []*yaml.RNode, r *types.Replacement) (*yaml.RNode, err
if r.Source.FieldPath == "" {
r.Source.FieldPath = types.DefaultReplacementFieldPath
}
fieldPath := kyaml_utils.SmarterPathSplitter(r.Source.FieldPath, ".")
fieldPath := utils.SmarterPathSplitter(r.Source.FieldPath, ".")
rn, err := source.Pipe(yaml.Lookup(fieldPath...))
if err != nil {
return nil, fmt.Errorf("error looking up replacement source: %w", err)
return nil, err
}
if rn.IsNilOrEmpty() {
return nil, fmt.Errorf("fieldPath `%s` is missing for replacement source %s", r.Source.FieldPath, r.Source.ResId)
if !rn.IsNilOrEmpty() {
return getRefinedValue(r.Source.Options, rn)
}
return getRefinedValue(r.Source.Options, rn)
}
// selectSourceNode finds the node that matches the selector, returning
// an error if multiple or none are found
func selectSourceNode(nodes []*yaml.RNode, selector *types.SourceSelector) (*yaml.RNode, error) {
var matches []*yaml.RNode
for _, n := range nodes {
ids, err := utils.MakeResIds(n)
if err != nil {
return nil, fmt.Errorf("error getting node IDs: %w", err)
}
for _, id := range ids {
if id.IsSelectedBy(selector.ResId) {
if len(matches) > 0 {
return nil, fmt.Errorf(
"multiple matches for selector %s", selector)
}
matches = append(matches, n)
break
}
}
}
if len(matches) == 0 {
return nil, fmt.Errorf("nothing selected by %s", selector)
}
return matches[0], nil
return rn, nil
}
func getRefinedValue(options *types.FieldOptions, rn *yaml.RNode) (*yaml.RNode, error) {
@@ -102,143 +147,35 @@ func getRefinedValue(options *types.FieldOptions, rn *yaml.RNode) (*yaml.RNode,
return n, nil
}
func applyReplacement(nodes []*yaml.RNode, value *yaml.RNode, targetSelectors []*types.TargetSelector) ([]*yaml.RNode, error) {
for _, selector := range targetSelectors {
if selector.Select == nil {
return nil, errors.Errorf("target must specify resources to select")
}
if len(selector.FieldPaths) == 0 {
selector.FieldPaths = []string{types.DefaultReplacementFieldPath}
}
for _, possibleTarget := range nodes {
ids, err := utils.MakeResIds(possibleTarget)
if err != nil {
return nil, err
}
// filter targets by label and annotation selectors
selectByAnnoAndLabel, err := selectByAnnoAndLabel(possibleTarget, selector)
if err != nil {
return nil, err
}
if !selectByAnnoAndLabel {
continue
}
// filter targets by matching resource IDs
for i, id := range ids {
if id.IsSelectedBy(selector.Select.ResId) && !rejectId(selector.Reject, &ids[i]) {
err := copyValueToTarget(possibleTarget, value, selector)
if err != nil {
return nil, err
}
break
}
// selectSourceNode finds the node that matches the selector, returning
// an error if multiple or none are found
func selectSourceNode(nodes []*yaml.RNode, selector *types.SourceSelector) (*yaml.RNode, error) {
var matches []*yaml.RNode
for _, n := range nodes {
if makeResId(n).IsSelectedBy(selector.ResId) {
if len(matches) > 0 {
return nil, fmt.Errorf(
"multiple matches for selector %s", selector)
}
matches = append(matches, n)
}
}
return nodes, nil
if len(matches) == 0 {
return nil, fmt.Errorf("nothing selected by %s", selector)
}
return matches[0], nil
}
func selectByAnnoAndLabel(n *yaml.RNode, t *types.TargetSelector) (bool, error) {
if matchesSelect, err := matchesAnnoAndLabelSelector(n, t.Select); !matchesSelect || err != nil {
return false, err
// makeResId makes a ResId from an RNode.
func makeResId(n *yaml.RNode) *resid.ResId {
apiVersion := n.Field(yaml.APIVersionField)
var group, version string
if apiVersion != nil {
group, version = resid.ParseGroupVersion(yaml.GetValue(apiVersion.Value))
}
for _, reject := range t.Reject {
if reject.AnnotationSelector == "" && reject.LabelSelector == "" {
continue
}
if m, err := matchesAnnoAndLabelSelector(n, reject); m || err != nil {
return false, err
}
return &resid.ResId{
Gvk: resid.Gvk{Group: group, Version: version, Kind: n.GetKind()},
Name: n.GetName(),
Namespace: n.GetNamespace(),
}
return true, nil
}
func matchesAnnoAndLabelSelector(n *yaml.RNode, selector *types.Selector) (bool, error) {
r := resource.Resource{RNode: *n}
annoMatch, err := r.MatchesAnnotationSelector(selector.AnnotationSelector)
if err != nil {
return false, err
}
labelMatch, err := r.MatchesLabelSelector(selector.LabelSelector)
if err != nil {
return false, err
}
return annoMatch && labelMatch, nil
}
func rejectId(rejects []*types.Selector, id *resid.ResId) bool {
for _, r := range rejects {
if !r.ResId.IsEmpty() && id.IsSelectedBy(r.ResId) {
return true
}
}
return false
}
func copyValueToTarget(target *yaml.RNode, value *yaml.RNode, selector *types.TargetSelector) error {
for _, fp := range selector.FieldPaths {
createKind := yaml.Kind(0) // do not create
if selector.Options != nil && selector.Options.Create {
createKind = value.YNode().Kind
}
targetFieldList, err := target.Pipe(&yaml.PathMatcher{
Path: kyaml_utils.SmarterPathSplitter(fp, "."),
Create: createKind})
if err != nil {
return errors.WrapPrefixf(err, fieldRetrievalError(fp, createKind != 0))
}
targetFields, err := targetFieldList.Elements()
if err != nil {
return errors.WrapPrefixf(err, fieldRetrievalError(fp, createKind != 0))
}
if len(targetFields) == 0 {
return errors.Errorf(fieldRetrievalError(fp, createKind != 0))
}
for _, t := range targetFields {
if err := setFieldValue(selector.Options, t, value); err != nil {
return err
}
}
}
return nil
}
func fieldRetrievalError(fieldPath string, isCreate bool) string {
if isCreate {
return fmt.Sprintf("unable to find or create field %q in replacement target", fieldPath)
}
return fmt.Sprintf("unable to find field %q in replacement target", fieldPath)
}
func setFieldValue(options *types.FieldOptions, targetField *yaml.RNode, value *yaml.RNode) error {
value = value.Copy()
if options != nil && options.Delimiter != "" {
if targetField.YNode().Kind != yaml.ScalarNode {
return fmt.Errorf("delimiter option can only be used with scalar nodes")
}
tv := strings.Split(targetField.YNode().Value, options.Delimiter)
v := yaml.GetValue(value)
// TODO: Add a way to remove an element
switch {
case options.Index < 0: // prefix
tv = append([]string{v}, tv...)
case options.Index >= len(tv): // suffix
tv = append(tv, v)
default: // replace an element
tv[options.Index] = v
}
value.YNode().Value = strings.Join(tv, options.Delimiter)
}
if targetField.YNode().Kind == yaml.ScalarNode {
// For scalar, only copy the value (leave any type intact to auto-convert int->string or string->int)
targetField.YNode().Value = value.YNode().Value
} else {
targetField.SetYNode(value.YNode())
}
return nil
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package replicacount
import (

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package replicacount
import (
@@ -17,17 +14,9 @@ import (
type Filter struct {
Replica types.Replica `json:"replica,omitempty" yaml:"replica,omitempty"`
FieldSpec types.FieldSpec `json:"fieldSpec,omitempty" yaml:"fieldSpec,omitempty"`
trackableSetter filtersutil.TrackableSetter
}
var _ kio.Filter = Filter{}
var _ kio.TrackableFilter = &Filter{}
// WithMutationTracker registers a callback which will be invoked each time a field is mutated
func (rc *Filter) WithMutationTracker(callback func(key, value, tag string, node *yaml.RNode)) {
rc.trackableSetter.WithMutationTracker(callback)
}
func (rc Filter) Filter(nodes []*yaml.RNode) ([]*yaml.RNode, error) {
return kio.FilterAll(yaml.FilterFunc(rc.run)).Filter(nodes)
@@ -44,5 +33,5 @@ func (rc Filter) run(node *yaml.RNode) (*yaml.RNode, error) {
}
func (rc Filter) set(node *yaml.RNode) error {
return rc.trackableSetter.SetEntry("", strconv.FormatInt(rc.Replica.Count, 10), yaml.NodeTagInt)(node)
return filtersutil.SetScalar(strconv.FormatInt(rc.Replica.Count, 10))(node)
}

View File

@@ -1,6 +1,3 @@
// Copyright 2022 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package replicacount
import (
@@ -10,17 +7,14 @@ import (
"github.com/stretchr/testify/assert"
filtertest_test "sigs.k8s.io/kustomize/api/testutils/filtertest"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/yaml"
)
func TestFilter(t *testing.T) {
mutationTrackerStub := filtertest_test.MutationTrackerStub{}
testCases := map[string]struct {
input string
expected string
filter Filter
mutationTracker func(key, value, tag string, node *yaml.RNode)
expectedSetValueArgs []filtertest_test.SetValueArg
input string
expected string
filter Filter
}{
"update field": {
input: `
@@ -167,44 +161,9 @@ spec:
FieldSpec: types.FieldSpec{Path: "spec/template/replicas"},
},
},
"mutation tracker": {
input: `
apiVersion: apps/v1
kind: Deployment
metadata:
name: dep
spec:
replicas: 5
`,
expected: `
apiVersion: apps/v1
kind: Deployment
metadata:
name: dep
spec:
replicas: 42
`,
filter: Filter{
Replica: types.Replica{
Name: "dep",
Count: 42,
},
FieldSpec: types.FieldSpec{Path: "spec/replicas"},
},
mutationTracker: mutationTrackerStub.MutationTracker,
expectedSetValueArgs: []filtertest_test.SetValueArg{
{
Value: "42",
Tag: "!!int",
NodePath: []string{"spec", "replicas"},
},
},
},
}
for tn, tc := range testCases {
mutationTrackerStub.Reset()
tc.filter.WithMutationTracker(tc.mutationTracker)
t.Run(tn, func(t *testing.T) {
if !assert.Equal(t,
strings.TrimSpace(tc.expected),
@@ -212,7 +171,6 @@ spec:
filtertest_test.RunFilter(t, tc.input, tc.filter))) {
t.FailNow()
}
assert.Equal(t, tc.expectedSetValueArgs, mutationTrackerStub.SetValueArgs())
})
}
}

View File

@@ -1,6 +0,0 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
// Package suffix contains a kio.Filter implementation of the kustomize
// SuffixTransformer.
package suffix

View File

@@ -1,47 +0,0 @@
// Copyright 2021 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
package suffix_test
import (
"bytes"
"log"
"os"
"sigs.k8s.io/kustomize/api/filters/suffix"
"sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/kio"
)
func ExampleFilter() {
err := kio.Pipeline{
Inputs: []kio.Reader{&kio.ByteReader{Reader: bytes.NewBufferString(`
apiVersion: example.com/v1
kind: Foo
metadata:
name: instance
---
apiVersion: example.com/v1
kind: Bar
metadata:
name: instance
`)}},
Filters: []kio.Filter{suffix.Filter{
Suffix: "-baz", FieldSpec: types.FieldSpec{Path: "metadata/name"}}},
Outputs: []kio.Writer{kio.ByteWriter{Writer: os.Stdout}},
}.Execute()
if err != nil {
log.Fatal(err)
}
// Output:
// apiVersion: example.com/v1
// kind: Foo
// metadata:
// name: instance-baz
// ---
// apiVersion: example.com/v1
// kind: Bar
// metadata:
// name: instance-baz
}

Some files were not shown because too many files have changed in this diff Show More