kubernetes-sigs/kustomize
1
0
Fork 0
mirror of https://github.com/kubernetes-sigs/kustomize.git synced 2026-05-17 18:25:26 +00:00
Code Issues Packages Projects Releases 1 Wiki Activity

Converge local and cloud gorelease scripts, improve release notes

Browse Source
This commit is contained in:
Katrina Verey
2022-03-25 14:29:27 -04:00
parent c4febc59d5
commit e86c479690
9 changed files with 159 additions and 815 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
.gitignore vendored
View File

@@ -27,3 +27,6 @@ site/public/
site/resources/
site/.hugo_build.lock
**/node_modules/
# goreleaser artifacts
**/dist/

291
releasing/README.md
View File

@@ -16,6 +16,25 @@
This document describes how to perform a [semver release]
of one of the several [Go modules] in this repository.
In this repo, releasing a module is accomplished by applying
a tag to the repo and pushing it upstream. A minor release
branch is also created as necessary to track patch releases.
A properly formatted tag (described below) contains
the module name and version.
Pushing the tag upstream will trigger [Google Cloud Build] to build a release
and make it available on the [release page].
Cloud build reads its instructions from the
[`cloudbuild.yaml`] file in this directory.
We use a Go program to make the tagging and branch
creation process less error prone.
See this [multi-module repo] tagging discussion
for an explanation of the path-like portion of these tags.
#### semver review
Go's [semver]-compatible version tags take the form `v{major}.{minor}.{patch}`:
@@ -141,7 +160,7 @@ go mod edit -require=sigs.k8s.io/kustomize/kyaml@$versionKyaml plugin/builtin/pa
Create the PR:
```
createBranch pinToKyaml "Pin to kyaml $versionKyaml"
createBranch pinToKyaml "Update kyaml to $versionKyaml"
```
```
createPr
@@ -201,7 +220,7 @@ gorepomod pin cmd/config --doIt
Create the PR:
```
createBranch pinToCmdConfig "Pin to cmd/config $versionCmdConfig" &&
createBranch pinToCmdConfig "Update cmd/config to $versionCmdConfig" &&
createPr
```
@@ -256,7 +275,7 @@ gorepomod pin api --doIt
Create the PR:
```
createBranch pinToApi "Pin to api $versionApi" &&
createBranch pinToApi "Update api to $versionApi" &&
createPr
```
@@ -383,7 +402,7 @@ Image sha256 can be found in the image registry in the GCP
project [k8s-staging-kustomize].
Commit and push your changes. Then create a PR to [k8s.io] to promote
new images. Assign the PR to @monopole and @Shell32-natsu.
the new image.
## Update kustomize-in-kubectl
@@ -410,267 +429,7 @@ https://github.com/kubernetes/kubernetes/pull/103419
https://github.com/kubernetes/kubernetes/pull/106389
----
Older notes follow:
## Public Modules
# Testing changes to the release pipeline
[`sigs.k8s.io/kustomize/api`]: #sigsk8siokustomizeapi
[`sigs.k8s.io/kustomize/kustomize`]: #sigsk8siokustomizekustomize
[`sigs.k8s.io/kustomize/kyaml`]: #sigsk8siokustomizekyaml
[`sigs.k8s.io/kustomize/cmd/config`]: #sigsk8siokustomizecmdconfig
[kustomize/v3.2.1]: /../../releases/tag/kustomize%2Fv3.2.1
| Module Name | Module Description | Example Tag | Example Branch Name |
| ------ | --- | --- | --- |
| [`sigs.k8s.io/kustomize/kustomize`] | kustomize executable | _kustomize/v3.2.2_ | _release-kustomize-v3.2.2_ |
| [`sigs.k8s.io/kustomize/api`] | kustomize API | _api/v0.1.0_ | _release-api-v0.1_ |
| [`sigs.k8s.io/kustomize/kyaml`] | k8s-specific yaml editting | _kyaml/v0.3.3_ | _release-kyaml-v0.2_ |
| [`sigs.k8s.io/kustomize/cmd/config`] | kyaml related commands | _cmd/config/v0.3.3_ | _release-cmd/config-v0.3_ |
### sigs.k8s.io/kustomize/kustomize
The `kustomize` program, a CLI for performing
structured edits to Kubernetes Resource Model (KRM) YAML.
There's only one public package in this module.
It's called `main`, it's therefore unimportable.
It holds the `kustomize` executable.
See the [installation instructions](../docs/INSTALL.md)
to perform an install.
### sigs.k8s.io/kustomize/api
The [kustomize Go API](https://github.com/kubernetes-sigs/kustomize/tree/master/api).
The packages in this module are used by API clients,
like the `kustomize` program itself, and programs in
other repositories, e.g. `kubectl`. They include
methods to read, edit and emit modified YAML.
Release notes should appear on the [release page].
The package has a toy executable called `api`, which,
if run, prints the API release version, but has no
other use.
### sigs.k8s.io/kustomize/kyaml
The [kyaml module](https://github.com/kubernetes-sigs/kustomize/tree/master/kyaml).
Low level packages for transforming YAML associated
with KRM objects.
These are used to build _kyaml filters_, computational units
that accept and emit KRM YAML, editing or simply validating it.
The kustomize `api` and the `cmd/config` packages are built on this.
### sigs.k8s.io/kustomize/cmd/config
The [cmd/config module](https://github.com/kubernetes-sigs/kustomize/tree/master/cmd/config).
A collection od CLI commands that correspond to
kyaml filters.
## Manual process
In this repo, releasing a module is accomplished by applying
a tag to the repo and pushing it upstream. A minor release
branch is also created as necessary to track patch releases.
A properly formatted tag (described below) contains
the module name and version.
Pushing the tag upstream will trigger [Google Cloud Build] to build a release
and make it available on the [release page].
Cloud build reads its instructions from the
[`cloudbuild.yaml`] file in this directory.
We use a Go program to make the tagging and branch
creation process less error prone.
See this [multi-module repo] tagging discussion
for an explanation of the path-like portion of these tags.
### Get up to date
It's assumed that whatever is already commited to the latest
commit is passing all tests and appropriate for release.
```
git fetch upstream
git checkout master
git rebase upstream/master
make prow-presubmit-check
```
### Select a module to release
E.g.
```
module=kustomize # The kustomize executable
module=api # The API
```
### Review tags to help determine new tag
Local:
```
git tag -l | grep $module
```
Remote:
```
git ls-remote --tags upstream | grep $module
```
Set the version you want:
```
major=0; minor=1; patch=0
```
### Create the release branch
Every module release occurs on it's own git branch.
The branch name doesn't include the patch number,
since the branch accumulates patch releases.
> TODO: define procedure for doing a cherrypick (committing a patch) to a
> release branch that already exists.
Name the branch:
```
branch="release-${module}-v${major}.${minor}"
echo "branch=$branch"
```
Create it:
```
git checkout -b $branch
```
### Define the release tag
```
tag="${module}/v${major}.${minor}.${patch}"
echo "tag=$tag"
```
### Pin modules to their dependencies.
This is achieved via a `replace` directive
in a module's `go.mod` file.
```
# Update the following as needed, obviously.
# git checkout -b pinTheRelease
# go mod edit -dropreplace=sigs.k8s.io/kustomize/api $module/go.mod
# go mod edit -require=sigs.k8s.io/kustomize/api@v0.1.1 $module/go.mod
# git commit -a -m "Drop API module replacement"
```
### Push the release branch
```
git push -f upstream $branch
```
#### if replacing a release...
Must delete the tag before re-pushing it.
Dangerous - only do this if you're sure nothing
has already pulled the release.
Delete the tag locally:
```
git tag --delete $tag
```
Delete it upstream:
```
# Disable push protection:
git remote set-url --push upstream git@github.com:kubernetes-sigs/kustomize.git
# The empty space before the colon effectively means delete the tag.
git push upstream :refs/tags/$tag
# Enable push protection:
git remote set-url --push upstream no_push
```
Optionally visit the [release page] and delete
(what has now become) the _draft_ release for that
version.
### Tag the local repository
```
git tag -a $tag -m "Release $tag on branch $branch"
```
Move the `latest_kustomize` tag:
```
git tag -d latest_kustomize
git push upstream :latest_kustomize
git tag -a latest_kustomize
```
### Optionally build locally
[localbuild.sh]: localbuild.sh
Load the same version of `goreleaser` referenced in `cloudbuild.yaml` via docker and run [localbuild.sh] from the container's command line:
```
# Get goreleaser image from cloudbuild.yaml
export GORELEASER_IMAGE=goreleaser/goreleaser:v0.172.1
# Drop into a shell
docker run -it --entrypoint=/bin/bash -v $(pwd):/go/src/github.com/kubernetes-sigs/kustomize -w /go/src/github.com/kubernetes-sigs/kustomize $GORELEASER_IMAGE
# Run build
./releasing/localbuild.sh TAG [--snapshot]
```
### Optionally build and release locally
[cloudbuild-local.sh]: cloudbuild-local.sh
Install [`cloud-build-local`], then run [cloudbuild-local.sh]:
```
./releasing/cloudbuild-local.sh $module
```
This should create release artifacts in a local directory.
### Trigger the cloud build by pushing the tag
Push the tag:
```
git remote set-url --push upstream git@github.com:kubernetes-sigs/kustomize.git
git push upstream $tag
git push upstream latest_kustomize
git remote set-url --push upstream no_push
```
This triggers [Google Cloud Build].
### Update release notes
Visit the [release page] and edit the release notes as desired.
You can test the release script locally by running [cloudbuild.sh](cloudbuild.sh) in a container or by installing Cloud Build Local and running [cloudbuild-local.sh](cloudbuild-local.sh). See each of those files for more details on their usage.

2
releasing/cloudbuild-local.sh
View File

@@ -27,7 +27,7 @@ cp releasing/cloudbuild.yaml $config
# Add the --snapshot flag to suppress the
# github release and leave the build output
# in the kustomize/dist directory.
sed -i "s|# - '--snapshot|- '--snapshot|" $config
sed -i "" "s|# - '--snapshot|- '--snapshot|" $config
echo "Executing cloud-build-local with config file $config :"
echo "========================="

131
releasing/cloudbuild.sh
View File

@@ -1,131 +1,32 @@
#!/bin/bash
#
# Usage (from top of repo):
# This script is called by Kustomize's Cloud Build release pipeline.
# It installs jq (required for release note construction)
# and then runs goreleaser (http://goreleaser.com).
#
# releasing/cloudbuild.sh TAG [--snapshot]
# To test it locally, run it in a goreleaser container:
#
# Where TAG is in the form
# # Get goreleaser image from cloudbuild.yaml
# export GORELEASER_IMAGE=goreleaser/goreleaser:v0.179.0
#
# api/v1.2.3
# kustomize/v1.2.3
# cmd/config/v1.2.3
# ... etc.
# # Drop into a shell
# docker run -it --entrypoint=/bin/bash -v $(pwd):/go/src/github.com/kubernetes-sigs/kustomize -w /go/src/github.com/kubernetes-sigs/kustomize $GORELEASER_IMAGE
#
# Cloud build should be configured to trigger on tags
# matching:
# # Run this script in the container, where $TAG is the tag to "release" (e.g. kyaml/v0.13.4)
# ./releasing/cloudbuild.sh $TAG --snapshot
#
# [\w/]+/v\d+\.\d+\.\d+
#
# This script runs goreleaser (http://goreleaser.com),
# presumably from a cloudbuild.yaml step that installed it.
set -e
set -x
fullTag=$1
shift
echo "fullTag=$fullTag"
remainingArgs="$@"
echo "Remaining args: $remainingArgs"
# Take everything before the last slash.
# This is expected to match $module.
module=${fullTag%/*}
echo "module=$module"
# Find previous tag that matches the tags module
prevTag=$(git tag -l "$module*" --sort=-version:refname --no-contains=$fullTag | head -n 1)
# Generate the changelog for this release
# using the last two tags for the module
changeLogFile=$(mktemp)
git log $prevTag..$fullTag \
--pretty=oneline \
--abbrev-commit --no-decorate --no-color --no-merges \
-- $module > $changeLogFile
echo "Release notes:"
cat $changeLogFile
# Take everything after the last slash.
# This should be something like "v1.2.3".
semVer=`echo $fullTag | sed "s|$module/||"`
echo "semVer=$semVer"
# This is probably a directory called /workspace
echo "pwd = $PWD"
# Sanity check
echo "### ls -las . ################################"
ls -las .
echo "###################################"
# CD into the module directory.
# This directory expected to contain a main.go, so there's
# no need for extra details in the `build` stanza below.
cd $module
skipBuild=true
if [[ "$module" == "kustomize" || "$module" == "pluginator" ]]; then
# If releasing a main program, don't skip the build.
skipBuild=false
if ! command -v jq &> /dev/null
then
# This assumes we are in an alpine container (which is the case for goreleaser images)
echo "Installing jq."
apk add jq --no-cache
fi
goReleaserConfigFile=$(mktemp)
cat <<EOF >$goReleaserConfigFile
project_name: $module
archives:
- name_template: "${module}_${semVer}_{{ .Os }}_{{ .Arch }}"
builds:
- skip: $skipBuild
ldflags: >
-s
-X sigs.k8s.io/kustomize/api/provenance.version={{.Version}}
-X sigs.k8s.io/kustomize/api/provenance.gitCommit={{.Commit}}
-X sigs.k8s.io/kustomize/api/provenance.buildDate={{.Date}}
goos:
- linux
- darwin
- windows
goarch:
- amd64
- arm64
- s390x
- ppc64le
checksum:
name_template: 'checksums.txt'
env:
- CGO_ENABLED=0
- GO111MODULE=on
release:
github:
owner: kubernetes-sigs
name: kustomize
draft: true
EOF
cat $goReleaserConfigFile
date
time /usr/local/bin/goreleaser release \
--debug \
--timeout 10m \
--parallelism 7 \
--config=$goReleaserConfigFile \
--release-notes=$changeLogFile \
--rm-dist \
--skip-validate $remainingArgs
date
./releasing/run-goreleaser.sh "$fullTag" release "$@"

5
releasing/cloudbuild.yaml
View File

@@ -1,3 +1,6 @@
# Cloud build should be configured to trigger with this configuration on tags matching:
# [\w/]+/v\d+\.\d+\.\d+
#
steps:
- name: 'bash'
args:
@@ -36,7 +39,7 @@ steps:
# Run goreleaser indirectly via a shell script
# to configure it properly.
- name: goreleaser/goreleaser:v0.172.1
- name: goreleaser/goreleaser:v0.179.0
timeout: 12m
entrypoint: /bin/sh
dir: myClone

53
releasing/compile-changelog.sh Executable file
View File

@@ -0,0 +1,53 @@
#!/bin/bash
#
# Builds a PR-oriented changelog from the git history for the given module.
#
# Usage (from top of repo):
#
# releasing/compile-changelog.sh MODULE TAG CHANGE_LOG_FILE
#
# Where TAG is in the form
#
# api/v1.2.3
# kustomize/v1.2.3
# cmd/config/v1.2.3
# ... etc.
#
set -o errexit
set -o nounset
set -o pipefail
if [[ -z "${1-}" ]] || [[ -z "${2-}" ]] || [[ -z "${3-}" ]]; then
echo "Usage: $0 <module> <fullTag> <changeLogFile>"
echo "Example: $0 kyaml kyaml/v0.13.4 changelog.txt"
exit 1
fi
module=$1
fullTag=$2
changeLogFile=$3
# Find previous tag that matches the tags module
prevTag=$(git tag -l "$module*" --sort=-version:refname --no-contains="$fullTag" | head -n 1)
commits=( $(git log "$prevTag".."$fullTag" \
--pretty=format:'%H' \
--abbrev-commit --no-decorate --no-color --no-merges \
-- "$module") )
# There is a 256 character limit on the query parameter for the GitHub API, so split into batches then deduplicate results
batchSize=5
results=""
for((i=0; i < ${#commits[@]}; i+=batchSize))
do
commitList=$(IFS="+"; echo "${commits[@]:i:batchSize}" | sed 's/ /+/g')
if newResults=$(curl -sSL "https://api.github.com/search/issues?q=$commitList+repo%3Akubernetes-sigs%2Fkustomize" | jq -r '[ .items[] | { number, title } ]'); then
results=$(echo "$results" "$newResults" | jq -s '.[0] + .[1] | unique')
else
echo "Failed to fetch results for commits: $commitList"
exit 1
fi
done
echo "${results}" | jq -r '.[] | select( .title | startswith("Back to development mode") | not) | "#\(.number): \(.title)" ' > "$changeLogFile"

88
releasing/localbuild.sh → releasing/run-goreleaser.sh
View File

@@ -1,10 +1,10 @@
#!/bin/bash
#
# Works exactly like cloudbuild.sh but doesn't perform a release.
# Builds and optionally releases the specified module
#
# Usage (from top of repo):
#
# releasing/localbuild.sh TAG [--snapshot]
# releasing/run-goreleaser.sh TAG MODE[build|release] [--snapshot]
#
# Where TAG is in the form
#
@@ -13,16 +13,30 @@
# cmd/config/v1.2.3
# ... etc.
#
# This script runs a build through goreleaser (http://goreleaser.com) but nothing else.
#
set -e
set -x
set -o errexit
set -o nounset
set -o pipefail
if [[ -z "${1-}" || -z "${2-}" ]]; then
echo "Usage: $0 TAG MODE [goreleaser flags]"
echo " TAG: the tag to build or release, e.g. api/v1.2.3"
echo " MODE: build or release"
exit 1
fi
fullTag=$1
shift
echo "fullTag=$fullTag"
if [[ $1 == "release" || $1 == "build" ]]; then
mode=$1
shift
else
echo "Error: mode must be build or release"
exit 1
fi
remainingArgs="$@"
echo "Remaining args: $remainingArgs"
@@ -31,38 +45,38 @@ echo "Remaining args: $remainingArgs"
module=${fullTag%/*}
echo "module=$module"
# Find previous tag that matches the tags module
prevTag=$(git tag -l "$module*" --sort=-version:refname --no-contains=$fullTag | head -n 1)
# Take everything after the last slash.
# This should be something like "v1.2.3".
semVer=${fullTag#$module/}
echo "semVer=$semVer"
# Generate the changelog for this release
# using the last two tags for the module
changeLogFile=$(mktemp)
git log $prevTag..$fullTag \
--pretty=oneline \
--abbrev-commit --no-decorate --no-color --no-merges \
-- $module > $changeLogFile
echo "Release notes:"
cat $changeLogFile
# Take everything after the last slash.
# This should be something like "v1.2.3".
semVer=`echo $fullTag | sed "s|$module/||"`
echo "semVer=$semVer"
./releasing/compile-changelog.sh "$module" "$fullTag" "$changeLogFile"
echo
echo "######### Release notes: ##########"
cat "$changeLogFile"
echo "###################################"
echo
# This is probably a directory called /workspace
echo "pwd = $PWD"
# Sanity check
echo "### ls -las . ################################"
echo
echo "############ DEBUG ##############"
echo "pwd = $PWD"
echo "ls -las ."
ls -las .
echo "###################################"
echo
# CD into the module directory.
# This directory expected to contain a main.go, so there's
# no need for extra details in the `build` stanza below.
cd $module
# This is used in goreleaser.yaml
skipBuild=true
if [[ "$module" == "kustomize" || "$module" == "pluginator" ]]; then
# If releasing a main program, don't skip the build.
@@ -112,16 +126,26 @@ release:
EOF
cat $goReleaserConfigFile
echo
echo "############# CONFIG ##############"
cat "$goReleaserConfigFile"
echo "###################################"
echo
args=(
--debug
--timeout 10m
--parallelism 7
--config="$goReleaserConfigFile"
--rm-dist
--skip-validate
)
if [[ $mode == "release" ]]; then
args+=(--release-notes="$changeLogFile")
fi
date
time /usr/local/bin/goreleaser build \
--debug \
--timeout 10m \
--parallelism 7 \
--config=$goReleaserConfigFile \
--rm-dist \
--skip-validate $remainingArgs
export PATH="/usr/local/bin:$PATH"
set -x
time goreleaser "$mode" "${args[@]}" $remainingArgs
date

281
releasing/vendor_kustomize.diff
View File

@@ -1,281 +0,0 @@
commit 1b893558aa83ac6491e5ba416b493170a9045fec
Author: Jingfang Liu <jingfangliu@google.com>
Date: Mon Nov 12 10:26:12 2018 -0800
last change
diff --git a/staging/src/k8s.io/cli-runtime/artifacts/kustomization/configMap.yaml b/staging/src/k8s.io/cli-runtime/artifacts/kustomization/configMap.yaml
new file mode 100644
index 0000000000..0008853094
--- /dev/null
+++ b/staging/src/k8s.io/cli-runtime/artifacts/kustomization/configMap.yaml
@@ -0,0 +1,8 @@
+
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: the-map
+data:
+ altGreeting: "Good Morning!"
+ enableRisky: "false"
diff --git a/staging/src/k8s.io/cli-runtime/artifacts/kustomization/deployment.yaml b/staging/src/k8s.io/cli-runtime/artifacts/kustomization/deployment.yaml
new file mode 100644
index 0000000000..6e79409080
--- /dev/null
+++ b/staging/src/k8s.io/cli-runtime/artifacts/kustomization/deployment.yaml
@@ -0,0 +1,30 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: the-deployment
+spec:
+ replicas: 3
+ template:
+ metadata:
+ labels:
+ deployment: hello
+ spec:
+ containers:
+ - name: the-container
+ image: monopole/hello:1
+ command: ["/hello",
+ "--port=8080",
+ "--enableRiskyFeature=$(ENABLE_RISKY)"]
+ ports:
+ - containerPort: 8080
+ env:
+ - name: ALT_GREETING
+ valueFrom:
+ configMapKeyRef:
+ name: the-map
+ key: altGreeting
+ - name: ENABLE_RISKY
+ valueFrom:
+ configMapKeyRef:
+ name: the-map
+ key: enableRisky
diff --git a/staging/src/k8s.io/cli-runtime/artifacts/kustomization/kustomization.yaml b/staging/src/k8s.io/cli-runtime/artifacts/kustomization/kustomization.yaml
new file mode 100644
index 0000000000..6e1e3202d5
--- /dev/null
+++ b/staging/src/k8s.io/cli-runtime/artifacts/kustomization/kustomization.yaml
@@ -0,0 +1,5 @@
+nameprefix: test-
+ resources:
+- deployment.yaml
+- service.yaml
+- configMap.yaml
diff --git a/staging/src/k8s.io/cli-runtime/artifacts/kustomization/service.yaml b/staging/src/k8s.io/cli-runtime/artifacts/kustomization/service.yaml
new file mode 100644
index 0000000000..2942cdb7df
--- /dev/null
+++ b/staging/src/k8s.io/cli-runtime/artifacts/kustomization/service.yaml
@@ -0,0 +1,13 @@
+kind: Service
+apiVersion: v1
+metadata:
+ name: the-service
+spec:
+ selector:
+ deployment: hello
+ type: LoadBalancer
+ ports:
+ - protocol: TCP
+ port: 8666
+ targetPort: 8080
+
\ No newline at end of file
diff --git a/staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/BUILD b/staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/BUILD
index 22b34de008..b91d1c0130 100644
--- a/staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/BUILD
+++ b/staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/BUILD
@@ -35,12 +35,15 @@ go_library(
"//staging/src/k8s.io/apimachinery/pkg/util/sets:go_default_library",
"//staging/src/k8s.io/apimachinery/pkg/util/yaml:go_default_library",
"//staging/src/k8s.io/apimachinery/pkg/watch:go_default_library",
+ "//staging/src/k8s.io/cli-runtime/pkg/kustomize/k8sdeps:go_default_library",
"//staging/src/k8s.io/client-go/discovery:go_default_library",
"//staging/src/k8s.io/client-go/kubernetes/scheme:go_default_library",
"//staging/src/k8s.io/client-go/rest:go_default_library",
"//staging/src/k8s.io/client-go/restmapper:go_default_library",
"//vendor/golang.org/x/text/encoding/unicode:go_default_library",
"//vendor/golang.org/x/text/transform:go_default_library",
+ "//vendor/sigs.k8s.io/kustomize/pkg/commands/build:go_default_library",
+ "//vendor/sigs.k8s.io/kustomize/pkg/fs:go_default_library",
],
)
diff --git a/staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/builder_test.go b/staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/builder_test.go
index 7fd526b33c..801f13f772 100644
--- a/staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/builder_test.go
+++ b/staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/builder_test.go
@@ -465,27 +465,48 @@ func TestPathBuilderWithMultipleInvalid(t *testing.T) {
}
func TestDirectoryBuilder(t *testing.T) {
- b := newDefaultBuilder().
- FilenameParam(false, &FilenameOptions{Recursive: false, Filenames: []string{"../../../artifacts/guestbook"}}).
- NamespaceParam("test").DefaultNamespace()
+ tests := []struct {
+ directories []string
+ singleItem bool
+ number int
+ expectedNames []string
+ }{
+ {[]string{"../../../artifacts/guestbook"}, false, 3, []string{"redis-master"}},
+ {[]string{"../../../artifacts/kustomization"}, true, 3, []string{"test-the-deployment"}},
+ {[]string{"../../../artifacts/guestbook", "../../../artifacts/kustomization"}, false, 6, []string{"redis-master", "test-the-deployment"}},
+ }
- test := &testVisitor{}
- singleItemImplied := false
+ for _, tt := range tests {
+ b := newDefaultBuilder().
+ FilenameParam(false, &FilenameOptions{Recursive: false, Filenames: tt.directories}).
+ NamespaceParam("test").DefaultNamespace()
- err := b.Do().IntoSingleItemImplied(&singleItemImplied).Visit(test.Handle)
- if err != nil || singleItemImplied || len(test.Infos) < 3 {
- t.Fatalf("unexpected response: %v %t %#v", err, singleItemImplied, test.Infos)
- }
+ test := &testVisitor{}
+ singleItemImplied := false
- found := false
- for _, info := range test.Infos {
- if info.Name == "redis-master" && info.Namespace == "test" && info.Object != nil {
- found = true
- break
+ err := b.Do().IntoSingleItemImplied(&singleItemImplied).Visit(test.Handle)
+ if err != nil || singleItemImplied != tt.singleItem || len(test.Infos) < tt.number {
+ t.Fatalf("unexpected response: %v %t %#v", err, singleItemImplied, test.Infos)
+ }
+
+ contained := func(name string) bool {
+ for _, info := range test.Infos {
+ if info.Name == name && info.Namespace == "test" && info.Object != nil {
+ return true
+ }
+ }
+ return false
+ }
+
+ allFound := true
+ for _, name := range tt.expectedNames {
+ if !contained(name) {
+ allFound = false
+ }
+ }
+ if !allFound {
+ t.Errorf("unexpected responses: %#v", test.Infos)
}
- }
- if !found {
- t.Errorf("unexpected responses: %#v", test.Infos)
}
}
diff --git a/staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/visitor.go b/staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/visitor.go
index 32c1a691a5..d7a37e1cde 100644
--- a/staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/visitor.go
+++ b/staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/visitor.go
@@ -20,10 +20,12 @@ import (
"bytes"
"fmt"
"io"
+ "io/ioutil"
"net/http"
"net/url"
"os"
"path/filepath"
+ "strings"
"time"
"golang.org/x/text/encoding/unicode"
@@ -38,6 +40,9 @@ import (
utilerrors "k8s.io/apimachinery/pkg/util/errors"
"k8s.io/apimachinery/pkg/util/yaml"
"k8s.io/apimachinery/pkg/watch"
+ "k8s.io/cli-runtime/pkg/kustomize/k8sdeps"
+ "sigs.k8s.io/kustomize/pkg/commands/build"
+ "sigs.k8s.io/kustomize/pkg/fs"
)
const (
@@ -452,7 +457,10 @@ func ExpandPathsToFileVisitors(mapper *mapper, paths string, recursive bool, ext
if err != nil {
return err
}
-
+ if isKustomizationDir(path) {
+ visitors = append(visitors, NewKustomizationVisitor(mapper, path, schema))
+ return filepath.SkipDir
+ }
if fi.IsDir() {
if path != paths && !recursive {
return filepath.SkipDir
@@ -463,7 +471,10 @@ func ExpandPathsToFileVisitors(mapper *mapper, paths string, recursive bool, ext
if path != paths && ignoreFile(path, extensions) {
return nil
}
-
+ if strings.HasSuffix(path, "kustomization.yaml") {
+ visitors = append(visitors, NewKustomizationVisitor(mapper, filepath.Dir(path), schema))
+ return nil
+ }
visitor := &FileVisitor{
Path: path,
StreamVisitor: NewStreamVisitor(nil, mapper, path, schema),
@@ -479,6 +490,14 @@ func ExpandPathsToFileVisitors(mapper *mapper, paths string, recursive bool, ext
return visitors, nil
}
+func isKustomizationDir(path string) bool {
+ if _, err := os.Stat(filepath.Join(path, "kustomization.yaml")); err == nil {
+ return true
+ }
+ return false
+}
+
+
// FileVisitor is wrapping around a StreamVisitor, to handle open/close files
type FileVisitor struct {
Path string
@@ -507,6 +526,37 @@ func (v *FileVisitor) Visit(fn VisitorFunc) error {
return v.StreamVisitor.Visit(fn)
}
+// KustomizationVisitor prorvides the output of kustomization build
+type KustomizationVisitor struct {
+ Path string
+ *StreamVisitor
+}
+
+// Visit in a KustomizationVisitor build the kustomization output
+func (v *KustomizationVisitor) Visit(fn VisitorFunc) error {
+ fSys := fs.MakeRealFS()
+ f := k8sdeps.NewFactory()
+ var out bytes.Buffer
+ cmd := build.NewCmdBuild(&out, fSys, f.ResmapF, f.TransformerF)
+ cmd.SetArgs([]string{v.Path})
+ // we want to silence usage, error output, and any future output from cobra
+ // we will get error output as a golang error from execute
+ cmd.SetOutput(ioutil.Discard)
+ _, err := cmd.ExecuteC()
+ if err != nil {
+ return err
+ }
+ v.StreamVisitor.Reader = bytes.NewReader(out.Bytes())
+ return v.StreamVisitor.Visit(fn)
+}
+
+func NewKustomizationVisitor(mapper *mapper, path string, schema ContentValidator) *KustomizationVisitor {
+ return &KustomizationVisitor{
+ Path: path,
+ StreamVisitor: NewStreamVisitor(nil, mapper, path, schema),
+ }
+}
+
// StreamVisitor reads objects from an io.Reader and walks them. A stream visitor can only be
// visited once.
// TODO: depends on objects being in JSON format before being passed to decode - need to implement

118
releasing/vendor_kustomize.sh
View File

@@ -1,118 +0,0 @@
#!/bin/bash
#
# Copyright 2018 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -e
set -x
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
# vendor_kustomize.sh creates the change in kubernetes repo for vendoring kustomize
function setUpWorkspace {
KPATH=~/kustomize_vendor
mkdir $KPATH
GOPATH=$KPATH
}
function cloneK8s {
mkdir -p $KPATH/src/k8s.io
cd $KPATH/src/k8s.io
git clone git@github.com:kubernetes/kubernetes.git
}
function godepRestore {
cd $KPATH/src/k8s.io/kubernetes
# restore dependencies
hack/run-in-gopath.sh hack/godep-restore.sh
}
function getKustomizeDeps {
# get Kustomize and Kustomize dependencies
hack/run-in-gopath.sh godep get sigs.k8s.io/kustomize/pkg/commands
hack/run-in-gopath.sh godep get github.com/bgentry/go-netrc/netrc
hack/run-in-gopath.sh godep get github.com/hashicorp/go-cleanhttp
hack/run-in-gopath.sh godep get github.com/hashicorp/go-getter
hack/run-in-gopath.sh godep get github.com/hashicorp/go-safetemp
hack/run-in-gopath.sh godep get github.com/hashicorp/go-version
# The hashes below passed bin/pre-commit.sh with kustomize HEAD at time of merger.
DEPS=(
"hashicorp/go-getter 4bda8fa99001c61db3cad96b421d4c12a81f256d"
"hashicorp/go-cleanhttp d5fe4b57a186c716b0e00b8c301cbd9b4182694d"
"hashicorp/go-safetemp b1a1dbde6fdc11e3ae79efd9039009e22d4ae240"
"hashicorp/go-version 270f2f71b1ee587f3b609f00f422b76a6b28f348"
"bgentry/go-netrc 9fd32a8b3d3d3f9d43c341bfe098430e07609480"
"mitchellh/go-homedir 58046073cbffe2f25d425fe1331102f55cf719de"
"mitchellh/go-testing-interface a61a99592b77c9ba629d254a693acffaeb4b7e28"
"ulikunitz/xz v0.5.4"
)
function foo {
cd $KPATH/src/k8s.io/kubernetes/_output/local/go/src/github.com/$1
git checkout $2
}
for i in "${DEPS[@]}"; do
foo $i
done
}
function updateK8s {
# Copy k8sdeps from Kustomize to cli-runtime in staging
mkdir -p $KPATH/src/k8s.io/kubernetes/staging/src/k8s.io/cli-runtime/pkg/kustomize
cp -r $KPATH/src/k8s.io/kubernetes/_output/local/go/src/sigs.k8s.io/kustomize/k8sdeps \
$KPATH/src/k8s.io/kubernetes/staging/src/k8s.io/cli-runtime/pkg/kustomize/k8sdeps
# Change import path of k8sdeps
find $KPATH/src/k8s.io/kubernetes/staging/src/k8s.io/cli-runtime/pkg/kustomize/k8sdeps \
-type f -name "*.go" | \
xargs sed -i \
's!sigs.k8s.io/kustomize/k8sdeps!k8s.io/cli-runtime/pkg/kustomize/k8sdeps!'
# Add kustomize command to kubectl
cp $DIR/vendor_kustomize.diff $KPATH/vendor_kustomize.diff
cd $GOPATH/src/k8s.io/kubernetes
git apply --ignore-space-change --ignore-whitespace $KPATH/vendor_kustomize.diff
}
function godepSave {
# Save all dependencies into k8s.io/kubernetes/vendor by running
# hack/godep-save.sh
hack/run-in-gopath.sh hack/godep-save.sh
}
function verify {
# make sure in k8s.io/kubernetes/vendor/sigs.k8s.io/kustomize
# there is no internal package
test 0 == $(ls $KPATH/src/k8s.io/kubernetes/vendor/sigs.k8s.io/kustomize | grep “internal” | wc -l)
# Make sure it compiles.
test 0 == $(bazel build cmd/kubectl:kubectl)
# next step, open a PR
echo "The change for vendoring kustomize is ready in $GOPATH/src/k8s.io/kubernetes.\n Next step, open a PR for it.\n"
}
setUpWorkspace
cloneK8s
godepRestore
getKustomizeDeps
updateK8s
godepSave
verify