kubernetes-sigs/kustomize
1
0
Fork 0
mirror of https://github.com/kubernetes-sigs/kustomize.git synced 2026-05-17 10:15:22 +00:00
Code Issues Packages Projects Releases 1 Wiki Activity

change_release_flow_to_build_scripts

Browse Source
This commit is contained in:
koba1t
2023-07-13 05:18:58 +09:00
parent 911ddcda40
commit b1717c8a97
8 changed files with 178 additions and 365 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
.github/workflows/release.yaml vendored Normal file
View File

@@ -0,0 +1,27 @@
name: release
on:
push:
tags:
- kyaml/v*
- cmd/config/v*
- api/v*
- kustomize/v*
jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Go 1.x
uses: actions/setup-go@v3
with:
go-version-file: go.work
id: go
- run: ./releasing/create-release.sh "${tag}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
tag: ${{ github.ref_name }}

44
releasing/README.md
View File

@@ -1,17 +1,20 @@
# Releasing
[release page]: /../../releases
[`cloud-build-local`]: https://github.com/GoogleCloudPlatform/cloud-build-local
[GitHub Actions]: /../../actions
[Google Cloud Build]: https://cloud.google.com/cloud-build
[semver]: https://semver.org
[Go modules]: https://github.com/golang/go/wiki/Modules
[multi-module repo]: https://github.com/go-modules-by-example/index/blob/master/009_submodules/README.md
[semver review]: #semver-review
[semver release]: #semver-review
[`cloudbuild.yaml`]: cloudbuild.yaml
[`cloudbuild_kustomize_image.yaml`]: cloudbuild_kustomize_image.yaml
[`release.yaml`]: ../.github/workflows/release.yaml
[`create-release.sh`]: create-release.sh
[kustomize repo release page]: https://github.com/kubernetes-sigs/kustomize/releases
[OpenAPI Readme]: ../kyaml/openapi/README.md
[project cloud build history page]: https://console.cloud.google.com/cloud-build/builds?project=k8s-staging-kustomize
[the build status for container image]: https://console.cloud.google.com/cloud-build/builds?project=k8s-staging-kustomize
[build history of GitHub Actions job]: /../../actions
This document describes how to perform a [semver release]
of one of the several [Go modules] in this repository.
@@ -23,11 +26,10 @@ branch is also created as necessary to track patch releases.
A properly formatted tag (described below) contains
the module name and version.
Pushing the tag upstream will trigger [Google Cloud Build] to build a release
and make it available on the [release page].
Pushing the tag upstream will trigger [GitHub Actions] to build a release and make it available on the [release page].
[GitHub Actions] reads its instructions from the [`release.yaml`] file in `.github/workflows` directory.
Cloud build reads its instructions from the
[`cloudbuild.yaml`] file in this directory.
And, container image contains `kustomize` binary will build [Google Cloud Build] that instructions from [`cloudbuild_kustomize_image.yaml`] file triggered by tags contain `kustomize` and release versions.
We use a Go program to make the tagging and branch
creation process less error prone.
@@ -123,7 +125,7 @@ testKustomizeRepo
While you're waiting for the tests, review the commit log:
```
releasing/compile-changelog.sh kyaml HEAD
releasing/compile-changelog.sh kyaml HEAD
```
Based on the changes to be included in this release, decide whether a patch, minor or major version bump is needed: [semver review].
@@ -144,8 +146,7 @@ Note the version:
versionKyaml=v0.10.20 # EDIT THIS!
```
See the process of the cloud build job
on the [project cloud build history page].
See the process of the [build history of GitHub Actions job].
Undraft the release on the [kustomize repo release page]:
* Make sure the version number is what you expect.
@@ -180,7 +181,7 @@ testKustomizeRepo
While you're waiting for the tests, review the commit log:
```
releasing/compile-changelog.sh cmd/config HEAD
releasing/compile-changelog.sh cmd/config HEAD
```
Based on the changes to be included in this release, decide whether a patch, minor or major version bump is needed: [semver review].
@@ -196,8 +197,7 @@ Note the version:
versionCmdConfig=v0.9.12 # EDIT THIS!
```
See the process of the cloud build job
on the [project cloud build history page].
See the process of the [build history of GitHub Actions job].
Undraft the release on the [kustomize repo release page]:
* Make sure the version number is what you expect.
@@ -233,7 +233,7 @@ testKustomizeRepo
While you're waiting for the tests, review the commit log:
```
releasing/compile-changelog.sh api HEAD
releasing/compile-changelog.sh api HEAD
```
Based on the changes to be included in this release, decide whether a patch, minor or major version bump is needed: [semver review].
@@ -249,8 +249,7 @@ Note the version:
versionApi=v0.8.10 # EDIT THIS!
```
See the process of the cloud build job
on the [project cloud build history page].
See the process of the [build history of GitHub Actions job].
Undraft the release on the [kustomize repo release page]:
* Make sure the version number is what you expect.
@@ -291,7 +290,7 @@ testKustomizeRepo
While you're waiting for the tests, review the commit log:
```
releasing/compile-changelog.sh kustomize HEAD
releasing/compile-changelog.sh kustomize HEAD
```
Based on the changes to be included in this release, decide whether a patch, minor or major version bump is needed: [semver review].
@@ -302,8 +301,9 @@ Based on the changes to be included in this release, decide whether a patch, min
gorepomod release kustomize [patch|minor|major] --doIt
```
See the process of the cloud build job
on the [project cloud build history page].
See the process of the [build history of GitHub Actions job].
And check the process of [the build status for container image].
Undraft the release on the [kustomize repo release page]:
* Make sure the version number is what you expect.
@@ -366,8 +366,7 @@ Checkout a new branch.
Edit file `registry.k8s.io/images/k8s-staging-kustomize/images.yaml`
to add the new kustomize version and the image sha256.
Image sha256 can be found in the image registry in the GCP
project [k8s-staging-kustomize].
Image sha256 can be found in the image registry in the GCP project [k8s-staging-kustomize].
Commit and push your changes. Then create a PR to [k8s.io] to promote
the new image.
@@ -400,4 +399,5 @@ https://github.com/kubernetes/kubernetes/pull/106389
# Testing changes to the release pipeline
You can test the release script locally by running [cloudbuild.sh](cloudbuild.sh) in a container or by installing Cloud Build Local and running [cloudbuild-local.sh](cloudbuild-local.sh). See each of those files for more details on their usage.
You can test the release script locally by running [`create-release.sh`].
See each of those files for more details on their usage.

57
releasing/cloudbuild-local.sh
View File

@@ -1,57 +0,0 @@
#!/bin/bash
# Copyright 2022 The Kubernetes Authors.
# SPDX-License-Identifier: Apache-2.0
#
# To test the release process, this script attempts
# to use a Google cloudbuild configuration to create
# release artifacts locally.
#
# See https://cloud.google.com/cloud-build/docs/build-debug-locally
#
# Usage: from the repo root, enter:
#
# ./releasing/cloudbuild-local.sh kustomize/v1.2.3
#
# or some other valid tag value.
#
# IMPORTANT:
# The process clones the repo at the given tag,
# so the repo must have the tag applied upstream.
# Either use an old tag, or disable the cloud build
# trigger so that a new testing tag can be applied
# without setting off a cloud build.
set -e
config=$(mktemp)
cp releasing/cloudbuild.yaml $config
# Add the --snapshot flag to suppress the
# github release and leave the build output
# in the kustomize/dist directory.
sed -i "" "s|# - '--snapshot|- '--snapshot|" $config
echo "Executing cloud-build-local with config file $config :"
echo "========================="
cat $config
echo "========================="
workspace=~/cloud-build-local-workspace
cloud-build-local \
--config=$config \
--substitutions=TAG_NAME=$1 \
--write-workspace=$workspace \
--dryrun=false \
.
# --bind-mount-source \
echo " "
echo "Result of local build:"
echo "##########################################"
tree ./$module/dist
echo "##########################################"
tree ./$workspace
echo "##########################################"

49
releasing/cloudbuild.sh
View File

@@ -1,49 +0,0 @@
#!/bin/bash
# Copyright 2022 The Kubernetes Authors.
# SPDX-License-Identifier: Apache-2.0
#
# This script is called by Kustomize's Cloud Build release pipeline.
# It installs jq (required for release note construction)
# and then runs goreleaser (http://goreleaser.com).
#
# To test it locally, run it in a goreleaser container:
#
# # Get build image from cloudbuild.yaml
# export GOLANG_IMAGE=golang:1.20
#
# # Drop into a shell
# docker run -it --entrypoint=/bin/bash -v $(pwd):/go/src/github.com/kubernetes-sigs/kustomize -w /go/src/github.com/kubernetes-sigs/kustomize $GOLANG_IMAGE
#
# # Run this script in the container, where $TAG is the tag to "release" (e.g. kyaml/v0.13.4)
# ./releasing/cloudbuild.sh $TAG --snapshot
#
set -o errexit
set -o nounset
set -o pipefail
if [[ -z "${1-}" ]] ; then
echo "Usage: $0 <fullTag> [--snapshot]"
echo "Example: $0 kyaml/v0.13.4"
exit 1
fi
set -x
fullTag=$1
shift
if ! command -v jq &> /dev/null
then
# This is expecting to be run from Cloud Build, in a Debian-based official golang container
echo "Installing jq."
apt-get update && apt-get install -y jq
fi
if ! command -v goreleaser &> /dev/null
then
echo "Installing goreleaser."
make --file Makefile-tools.mk "$(go env GOPATH)/bin/goreleaser"
fi
./releasing/run-goreleaser.sh "$fullTag" release "$@"

71
releasing/cloudbuild.yaml
View File

@@ -1,71 +0,0 @@
# Cloud build should be configured to trigger with this configuration on tags matching:
# [\w/]+/v\d+\.\d+\.\d+
#
steps:
- name: 'bash'
args:
- 'echo'
- 'Cloud build substitution check: '
- 'BUILD_ID=$BUILD_ID'
- 'PROJECT_ID=$PROJECT_ID'
- 'REVISION_ID=$REVISION_ID'
- 'REPO_NAME=$REPO_NAME'
- 'COMMIT_SHA=$COMMIT_SHA'
- 'BRANCH_NAME=$BRANCH_NAME'
- 'TAG_NAME=$TAG_NAME'
# Cloud build has already copied the repo at the tag that
# that triggered the build to its /workspace directory, but
# hasn't actually _cloned_ the repo (there's no .git directory).
#
# The goreleaser tool, however, needs the repo and its history
# to produce release notes.
#
# So clone the repo to /workspace/myClone to avoid directory
# name collision.
#
- name: gcr.io/cloud-builders/git
args:
- clone
- https://github.com/kubernetes-sigs/kustomize.git
- myClone
# Checkout the proper tag.
- name: gcr.io/cloud-builders/git
dir: myClone
args:
- checkout
- $TAG_NAME
# Run goreleaser indirectly via a shell script
# to configure it properly.
- name: golang:1.20
entrypoint: /bin/bash
dir: myClone
secretEnv: ['GITHUB_TOKEN']
env:
- 'GITHUB_USERNAME=KnVerey' # used to make authenticated curl requests to Github in releasing/compile-changelog.sh
args:
- releasing/cloudbuild.sh
- $TAG_NAME
# - '--snapshot'
# Use this final arg in a local build, to suppress
# the release and leave the 'dist' directory in place.
# Overall timeout
timeout: 14m
# golreleaser expects the GITHUB_TOKEN env var to hold the github token
# it needs to write the released package and notes back to github.
# The raw token was encrypted by `gcloud kms encrypt` (Key Management Service)
# The base64 of that is shown below. It's decrypted by cloud build
# and provided back to goreleaser.
# IMPORTANT: make sure the token does not end with a newline when you encrypt it!
# IMPORTANT: update the GITHUB_USERNAME env var above to match the github user whose token this is
secrets:
- kmsKeyName: projects/k8s-staging-kustomize/locations/global/keyRings/kust-cloud-key-ring/cryptoKeys/kust-cloud-key-name
secretEnv:
GITHUB_TOKEN: CiQAJ+XRLwPj71lnT8zn0UdE7ihQIdCzDUsgEX7+mRN4aJ2ffRUSUQAdel1M9mEzxqs6gln1dzoZkNU3lmh7ya0EY3i3zkyz0jJ7Qok6TZsp29dl2lRnza3KxVGTWXo6YHa2Z5Qe7RwgoQxdwSdR3GWLu4fm1h4aXA==
options:
machineType: 'N1_HIGHCPU_8'

24
releasing/compile-changelog.sh
View File

@@ -27,13 +27,6 @@ if [[ -z "${1-}" ]] || [[ -z "${2-}" ]]; then
exit 1
fi
if [[ -z "${GITHUB_USERNAME-}" ]] || [[ -z "${GITHUB_TOKEN-}" ]]; then
echo "WARNING: Please set GITHUB_USERNAME and GITHUB_TOKEN to avoid GitHub API rate limits."
github_auth_string=""
else
github_auth_string="-u ${GITHUB_USERNAME}:${GITHUB_TOKEN}"
fi
module=$1
fullTag=$2
changeLogFile="${3:-}"
@@ -57,12 +50,21 @@ for((i=0; i < ${#commits[@]}; i+=batchSize))
do
commitList=$(IFS="+"; echo "${commits[@]:i:batchSize}" | sed 's/ /+/g')
if ! newResultsRaw=$(curl -sSL "https://api.github.com/search/issues?q=$commitList+repo%3Akubernetes-sigs%2Fkustomize+is:pull-request" $github_auth_string); then
echo "Failed to fetch results for commits (exit code $?): $commitList"
exit 1
if [[ -z "${GITHUB_TOKEN-}" ]]; then
echo "WARNING: Please set GITHUB_TOKEN to avoid GitHub API rate limits."
if ! newResultsRaw=$(curl -sSL "https://api.github.com/search/issues?q=$commitList+repo%3Akubernetes-sigs%2Fkustomize+is:pull-request"); then
echo "Failed to fetch results for commits (exit code $?): $commitList"
exit 1
fi
else
if ! newResultsRaw=$(curl -sSL "https://api.github.com/search/issues?q=$commitList+repo%3Akubernetes-sigs%2Fkustomize+is:pull-request" -H "Authorization: Bearer $GITHUB_TOKEN"); then
echo "Failed to fetch results for commits (exit code $?): $commitList"
exit 1
fi
fi
if [[ "${newResultsRaw}" == *"API rate limit exceeded"* ]]; then
echo "GitHub API rate limit exceeded. Please set GITHUB_USERNAME and GITHUB_TOKEN to avoid this."
echo "GitHub API rate limit exceeded. Please set GITHUB_TOKEN to avoid this."
exit 1
fi

116
releasing/create-release.sh Executable file
View File

@@ -0,0 +1,116 @@
#!/bin/bash
# Copyright 2023 The Kubernetes Authors.
# SPDX-License-Identifier: Apache-2.0
#
# This script is called by Kustomize's release pipeline.
# It needs jq (required for release note construction) and [GitHub CLI](https://cli.github.com/).
#
# To test it locally:
#
# # Please install jq and GitHub CLI. (e.g. macOS)
# brew install jq gh
#
# # Setup GitHub CLI
# gh auth login
#
# # Run this script, where $TAG is the tag to "release" (e.g. kyaml/v0.13.4)
# ./releasing/create-release.sh $TAG
#
# # Please remove Draft Release created by this script.
set -o errexit
set -o nounset
set -o pipefail
if [[ -z "${1-}" ]]; then
echo "Usage: $0 TAG"
echo " TAG: the tag to build or release, e.g. api/v1.2.3"
exit 1
fi
git_tag=$1
echo "release tag: $git_tag"
# Build the release binaries for every OS/arch combination.
# It builds compressed artifacts on $release_dir.
function build_kustomize_binary {
echo "build kustomize binaries"
version=$1
release_dir=$2
echo "build release artifacts to $release_dir"
mkdir -p "output"
# build date in ISO8601 format
build_date=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
for os in linux darwin windows; do
arch_list=(amd64 arm64)
if [ "$os" == "linux" ]; then
arch_list=(amd64 arm64 s390x ppc64le)
fi
for arch in "${arch_list[@]}" ; do
echo "Building $os-$arch"
# CGO_ENABLED=0 GOWORK=off GOOS=$os GOARCH=$arch go build -o output/kustomize -ldflags\
CGO_ENABLED=0 GOOS=$os GOARCH=$arch go build -o output/kustomize -ldflags\
"-s -w\
-X sigs.k8s.io/kustomize/api/provenance.version=$version\
-X sigs.k8s.io/kustomize/api/provenance.gitCommit=$(git rev-parse HEAD)\
-X sigs.k8s.io/kustomize/api/provenance.buildDate=$build_date"\
kustomize/main.go
if [ "$os" == "windows" ]; then
zip "${release_dir}/kustomize_${version}_${os}_${arch}.zip" output/kustomize
else
tar cvfz "${release_dir}/kustomize_${version}_${os}_${arch}.tar.gz" output/kustomize
fi
rm output/kustomize
done
rmdir output
done
}
function create_release {
git_tag=$1
# Take everything before the last slash.
# This is expected to match $module.
module=${git_tag%/*}
# Take everything after the last slash.
version=${git_tag##*/}
# Generate the changelog for this release
# using the last two tags for the module
changelog_file=$(mktemp)
./releasing/compile-changelog.sh "$module" "$git_tag" "$changelog_file"
additional_release_artifacts_arg=""
# build `kustomize` binary
if [[ "$module" == "kustomize" ]]; then
release_artifact_dir=$(mktemp -d)
build_kustomize_binary "$version" "$release_artifact_dir"
# additional_release_artifacts_arg+="$release_artifact_dir/*"
additional_release_artifacts_arg=("$release_artifact_dir"/*)
# create github releases
gh release create "$git_tag" \
--title "$git_tag"\
--draft \
--notes-file "$changelog_file"\
"${additional_release_artifacts_arg[@]}"
return
fi
# create github releases
gh release create "$git_tag" \
--title "$git_tag"\
--draft \
--notes-file "$changelog_file"
}
## create release
create_release "$git_tag"

155
releasing/run-goreleaser.sh
View File

@@ -1,155 +0,0 @@
#!/bin/bash
# Copyright 2022 The Kubernetes Authors.
# SPDX-License-Identifier: Apache-2.0
#
# Builds and optionally releases the specified module
#
# Usage (from top of repo):
#
# releasing/run-goreleaser.sh TAG MODE[build|release] [--snapshot]
#
# Where TAG is in the form
#
# api/v1.2.3
# kustomize/v1.2.3
# cmd/config/v1.2.3
# ... etc.
#
set -o errexit
set -o nounset
set -o pipefail
if [[ -z "${1-}" || -z "${2-}" ]]; then
echo "Usage: $0 TAG MODE [goreleaser flags]"
echo " TAG: the tag to build or release, e.g. api/v1.2.3"
echo " MODE: build or release"
exit 1
fi
fullTag=$1
shift
echo "fullTag=$fullTag"
export GORELEASER_CURRENT_TAG=$fullTag
if [[ $1 == "release" || $1 == "build" ]]; then
mode=$1
shift
else
echo "Error: mode must be build or release"
exit 1
fi
remainingArgs="$@"
echo "Remaining args: $remainingArgs"
# Take everything before the last slash.
# This is expected to match $module.
module=${fullTag%/*}
echo "module=$module"
# Take everything after the last slash.
# This should be something like "v1.2.3".
semVer=${fullTag#$module/}
echo "semVer=$semVer"
# Generate the changelog for this release
# using the last two tags for the module
changeLogFile=$(mktemp)
./releasing/compile-changelog.sh "$module" "$fullTag" "$changeLogFile"
echo
echo "######### Release notes: ##########"
cat "$changeLogFile"
echo "###################################"
echo
# This is probably a directory called /workspace
# Sanity check
echo
echo "############ DEBUG ##############"
echo "pwd = $PWD"
echo "ls -las ."
ls -las .
echo "###################################"
echo
# CD into the module directory.
# This directory expected to contain a main.go, so there's
# no need for extra details in the `build` stanza below.
cd $module
# This is used in goreleaser.yaml
skipBuild=true
if [[ "$module" == "kustomize" || "$module" == "pluginator" ]]; then
# If releasing a main program, don't skip the build.
skipBuild=false
fi
goReleaserConfigFile=$(mktemp)
cat <<EOF >$goReleaserConfigFile
project_name: $module
archives:
- name_template: "${module}_${semVer}_{{ .Os }}_{{ .Arch }}"
builds:
- skip: $skipBuild
ldflags: >
-s
-X sigs.k8s.io/kustomize/api/provenance.version={{.Version}}
-X sigs.k8s.io/kustomize/api/provenance.buildDate={{.Date}}
goos:
- linux
- darwin
- windows
goarch:
- amd64
- arm64
- s390x
- ppc64le
checksum:
name_template: 'checksums.txt'
env:
- CGO_ENABLED=0
- GO111MODULE=on
- GOWORK=off
release:
github:
owner: kubernetes-sigs
name: kustomize
draft: true
EOF
echo
echo "############# CONFIG ##############"
cat "$goReleaserConfigFile"
echo "###################################"
echo
args=(
--debug
--timeout 10m
--parallelism 7
--config="$goReleaserConfigFile"
--rm-dist
--skip-validate
)
if [[ $mode == "release" ]]; then
args+=(--release-notes="$changeLogFile")
fi
date
export PATH="/usr/local/bin:$PATH"
set -x
time goreleaser "$mode" "${args[@]}" $remainingArgs
date