change_release_flow_to_build_scripts

.github/workflows/release.yaml

@@ -0,0 +1,27 @@
+name: release
+
+on:
+  push:
+    tags:
+      - kyaml/v*
+      - cmd/config/v*
+      - api/v*
+      - kustomize/v*
+
+jobs:
+  release:
+    runs-on: ubuntu-latest
+    steps:
+    - name: Check out code into the Go module directory
+      uses: actions/checkout@v3
+      with:
+        fetch-depth: 0
+    - name: Set up Go 1.x
+      uses: actions/setup-go@v3
+      with:
+        go-version-file: go.work
+      id: go
+    - run: ./releasing/create-release.sh "${tag}"
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        tag: ${{ github.ref_name }}

releasing/README.md

@@ -1,17 +1,20 @@
 # Releasing
 
 [release page]: /../../releases
-[`cloud-build-local`]: https://github.com/GoogleCloudPlatform/cloud-build-local
+[GitHub Actions]: /../../actions
 [Google Cloud Build]: https://cloud.google.com/cloud-build
 [semver]: https://semver.org
 [Go modules]: https://github.com/golang/go/wiki/Modules
 [multi-module repo]: https://github.com/go-modules-by-example/index/blob/master/009_submodules/README.md
 [semver review]: #semver-review
 [semver release]: #semver-review
-[`cloudbuild.yaml`]: cloudbuild.yaml
+[`cloudbuild_kustomize_image.yaml`]: cloudbuild_kustomize_image.yaml
+[`release.yaml`]: ../.github/workflows/release.yaml
+[`create-release.sh`]: create-release.sh
 [kustomize repo release page]: https://github.com/kubernetes-sigs/kustomize/releases
 [OpenAPI Readme]: ../kyaml/openapi/README.md
-[project cloud build history page]: https://console.cloud.google.com/cloud-build/builds?project=k8s-staging-kustomize
+[the build status for container image]: https://console.cloud.google.com/cloud-build/builds?project=k8s-staging-kustomize
+[build history of GitHub Actions job]: /../../actions
 
 This document describes how to perform a [semver release]
 of one of the several [Go modules] in this repository.
@@ -23,11 +26,10 @@ branch is also created as necessary to track patch releases.
 A properly formatted tag (described below) contains
 the module name and version.
 
-Pushing the tag upstream will trigger [Google Cloud Build] to build a release
+Pushing the tag upstream will trigger [GitHub Actions] to build a release and make it available on the  [release page].
-and make it available on the  [release page].
+[GitHub Actions] reads its instructions from the [`release.yaml`] file in `.github/workflows` directory.
 
-Cloud build reads its instructions from the
+And, container image contains `kustomize` binary will build [Google Cloud Build] that instructions from [`cloudbuild_kustomize_image.yaml`] file triggered by tags contain `kustomize` and release versions.
-[`cloudbuild.yaml`] file in this directory.
 
 We use a Go program to make the tagging and branch
 creation process less error prone.
@@ -144,8 +146,7 @@ Note the version:
 versionKyaml=v0.10.20   # EDIT THIS!
 ```
 
-See the process of the cloud build job
+See the process of the [build history of GitHub Actions job].
-on the [project cloud build history page].
 
 Undraft the release on the [kustomize repo release page]:
 * Make sure the version number is what you expect.
@@ -196,8 +197,7 @@ Note the version:
 versionCmdConfig=v0.9.12 # EDIT THIS!
 ```
 
-See the process of the cloud build job
+See the process of the [build history of GitHub Actions job].
-on the [project cloud build history page].
 
 Undraft the release on the [kustomize repo release page]:
 * Make sure the version number is what you expect.
@@ -249,8 +249,7 @@ Note the version:
 versionApi=v0.8.10 # EDIT THIS!
 ```
 
-See the process of the cloud build job
+See the process of the [build history of GitHub Actions job].
-on the [project cloud build history page].
 
 Undraft the release on the [kustomize repo release page]:
 * Make sure the version number is what you expect.
@@ -302,8 +301,9 @@ Based on the changes to be included in this release, decide whether a patch, min
 gorepomod release kustomize [patch|minor|major] --doIt
 ```
 
-See the process of the cloud build job
+See the process of the [build history of GitHub Actions job].
-on the [project cloud build history page].
+
+And check the process of [the build status for container image].
 
 Undraft the release on the [kustomize repo release page]:
 * Make sure the version number is what you expect.
@@ -366,8 +366,7 @@ Checkout a new branch.
 Edit file `registry.k8s.io/images/k8s-staging-kustomize/images.yaml`
 to add the new kustomize version and the image sha256.
 
-Image sha256 can be found in the image registry in the GCP 
+Image sha256 can be found in the image registry in the GCP project [k8s-staging-kustomize].
-project [k8s-staging-kustomize].
 
 Commit and push your changes. Then create a PR to [k8s.io] to promote
 the new image.
@@ -400,4 +399,5 @@ https://github.com/kubernetes/kubernetes/pull/106389
 
 # Testing changes to the release pipeline
 
-You can test the release script locally by running [cloudbuild.sh](cloudbuild.sh) in a container or by installing Cloud Build Local and running [cloudbuild-local.sh](cloudbuild-local.sh). See each of those files for more details on their usage.
+You can test the release script locally by running [`create-release.sh`].
+See each of those files for more details on their usage.

releasing/cloudbuild-local.sh

@@ -1,57 +0,0 @@
-#!/bin/bash
-# Copyright 2022 The Kubernetes Authors.
-# SPDX-License-Identifier: Apache-2.0
-
-#
-# To test the release process, this script attempts
-# to use a Google cloudbuild configuration to create
-# release artifacts locally.
-#
-# See https://cloud.google.com/cloud-build/docs/build-debug-locally
-#
-# Usage: from the repo root, enter:
-#
-#   ./releasing/cloudbuild-local.sh kustomize/v1.2.3
-#
-# or some other valid tag value.
-#
-# IMPORTANT:
-#   The process clones the repo at the given tag,
-#   so the repo must have the tag applied upstream.
-#   Either use an old tag, or disable the cloud build
-#   trigger so that a new testing tag can be applied
-#   without setting off a cloud build.
-
-set -e
-
-config=$(mktemp)
-cp releasing/cloudbuild.yaml $config
-
-# Add the --snapshot flag to suppress the
-# github release and leave the build output
-# in the kustomize/dist directory.
-sed -i "" "s|# - '--snapshot|- '--snapshot|" $config
-
-echo "Executing cloud-build-local with config file $config :"
-echo "========================="
-cat $config
-echo "========================="
-
-workspace=~/cloud-build-local-workspace
-
-cloud-build-local \
-    --config=$config \
-    --substitutions=TAG_NAME=$1 \
-    --write-workspace=$workspace \
-    --dryrun=false \
-    .
-
-#  --bind-mount-source \
-
-echo " "
-echo "Result of local build:"
-echo "##########################################"
-tree ./$module/dist
-echo "##########################################"
-tree ./$workspace
-echo "##########################################"

releasing/cloudbuild.sh

@@ -1,49 +0,0 @@
-#!/bin/bash
-# Copyright 2022 The Kubernetes Authors.
-# SPDX-License-Identifier: Apache-2.0
-
-#
-# This script is called by Kustomize's Cloud Build release pipeline.
-# It installs jq (required for release note construction)
-# and then runs goreleaser (http://goreleaser.com).
-#
-# To test it locally, run it in a goreleaser container:
-#
-#   # Get build image from cloudbuild.yaml
-#   export GOLANG_IMAGE=golang:1.20
-#
-#   # Drop into a shell
-#   docker run -it --entrypoint=/bin/bash -v $(pwd):/go/src/github.com/kubernetes-sigs/kustomize -w /go/src/github.com/kubernetes-sigs/kustomize $GOLANG_IMAGE
-#
-#   # Run this script in the container, where $TAG is the tag to "release" (e.g. kyaml/v0.13.4)
-#   ./releasing/cloudbuild.sh $TAG --snapshot
-#
-
-set -o errexit
-set -o nounset
-set -o pipefail
-
-if [[ -z "${1-}" ]] ; then
-  echo "Usage: $0 <fullTag> [--snapshot]"
-  echo "Example: $0 kyaml/v0.13.4"
-  exit 1
-fi
-
-set -x
-fullTag=$1
-shift
-
-if ! command -v jq &> /dev/null
-then
-    # This is expecting to be run from Cloud Build, in a Debian-based official golang container
-    echo "Installing jq."
-    apt-get update && apt-get install -y jq
-fi
-
-if ! command -v goreleaser &> /dev/null
-then
-    echo "Installing goreleaser."
-    make --file Makefile-tools.mk "$(go env GOPATH)/bin/goreleaser"
-fi
-
-./releasing/run-goreleaser.sh "$fullTag" release "$@"

releasing/cloudbuild.yaml

@@ -1,71 +0,0 @@
-# Cloud build should be configured to trigger with this configuration on tags matching:
-#   [\w/]+/v\d+\.\d+\.\d+
-#
-steps:
-- name: 'bash'
-  args:
-  - 'echo'
-  - 'Cloud build substitution check: '
-  - 'BUILD_ID=$BUILD_ID'
-  - 'PROJECT_ID=$PROJECT_ID'
-  - 'REVISION_ID=$REVISION_ID'
-  - 'REPO_NAME=$REPO_NAME'
-  - 'COMMIT_SHA=$COMMIT_SHA'
-  - 'BRANCH_NAME=$BRANCH_NAME'
-  - 'TAG_NAME=$TAG_NAME'
-
-# Cloud build has already copied the repo at the tag that
-# that triggered the build to its /workspace directory, but
-# hasn't actually _cloned_ the repo (there's no .git directory).
-#
-# The goreleaser tool, however, needs the repo and its history
-# to produce release notes.
-#
-# So clone the repo to /workspace/myClone to avoid directory
-# name collision.
-#
-- name: gcr.io/cloud-builders/git
-  args:
-  - clone
-  - https://github.com/kubernetes-sigs/kustomize.git
-  - myClone
-
-# Checkout the proper tag.
-- name: gcr.io/cloud-builders/git
-  dir: myClone
-  args:
-  - checkout
-  - $TAG_NAME
-
-# Run goreleaser indirectly via a shell script
-# to configure it properly.
-- name: golang:1.20
-  entrypoint: /bin/bash
-  dir: myClone
-  secretEnv: ['GITHUB_TOKEN']
-  env:
-  - 'GITHUB_USERNAME=KnVerey' # used to make authenticated curl requests to Github in releasing/compile-changelog.sh
-  args:
-  - releasing/cloudbuild.sh
-  - $TAG_NAME
-  # - '--snapshot'
-  # Use this final arg in a local build, to suppress
-  # the release and leave the 'dist' directory in place.
-
-# Overall timeout
-timeout: 14m
-
-# golreleaser expects the GITHUB_TOKEN env var to hold the github token
-# it needs to write the released package and notes back to github.
-# The raw token was encrypted by `gcloud kms encrypt` (Key Management Service)
-# The base64 of that is shown below.  It's decrypted by cloud build
-# and provided back to goreleaser.
-# IMPORTANT: make sure the token does not end with a newline when you encrypt it!
-# IMPORTANT: update the GITHUB_USERNAME env var above to match the github user whose token this is
-secrets:
-- kmsKeyName: projects/k8s-staging-kustomize/locations/global/keyRings/kust-cloud-key-ring/cryptoKeys/kust-cloud-key-name
-  secretEnv:
-    GITHUB_TOKEN: CiQAJ+XRLwPj71lnT8zn0UdE7ihQIdCzDUsgEX7+mRN4aJ2ffRUSUQAdel1M9mEzxqs6gln1dzoZkNU3lmh7ya0EY3i3zkyz0jJ7Qok6TZsp29dl2lRnza3KxVGTWXo6YHa2Z5Qe7RwgoQxdwSdR3GWLu4fm1h4aXA==
-
-options:
-  machineType: 'N1_HIGHCPU_8'

releasing/compile-changelog.sh

@@ -27,13 +27,6 @@ if [[ -z "${1-}" ]] || [[ -z "${2-}" ]]; then
   exit 1
 fi
 
-if [[ -z "${GITHUB_USERNAME-}" ]] || [[ -z "${GITHUB_TOKEN-}" ]]; then
-  echo "WARNING: Please set GITHUB_USERNAME and GITHUB_TOKEN to avoid GitHub API rate limits."
-  github_auth_string=""
-else
-  github_auth_string="-u ${GITHUB_USERNAME}:${GITHUB_TOKEN}"
-fi
-
 module=$1
 fullTag=$2
 changeLogFile="${3:-}"
@@ -57,12 +50,21 @@ for((i=0; i < ${#commits[@]}; i+=batchSize))
 do
   commitList=$(IFS="+"; echo "${commits[@]:i:batchSize}" | sed 's/ /+/g')
 
-  if ! newResultsRaw=$(curl -sSL "https://api.github.com/search/issues?q=$commitList+repo%3Akubernetes-sigs%2Fkustomize+is:pull-request" $github_auth_string); then
+  if [[ -z "${GITHUB_TOKEN-}" ]]; then
-    echo "Failed to fetch results for commits (exit code $?): $commitList"
+    echo "WARNING: Please set GITHUB_TOKEN to avoid GitHub API rate limits."
-    exit 1
+    if ! newResultsRaw=$(curl -sSL "https://api.github.com/search/issues?q=$commitList+repo%3Akubernetes-sigs%2Fkustomize+is:pull-request"); then
+      echo "Failed to fetch results for commits (exit code $?): $commitList"
+      exit 1
+    fi
+  else
+    if ! newResultsRaw=$(curl -sSL "https://api.github.com/search/issues?q=$commitList+repo%3Akubernetes-sigs%2Fkustomize+is:pull-request" -H "Authorization: Bearer $GITHUB_TOKEN"); then
+      echo "Failed to fetch results for commits (exit code $?): $commitList"
+      exit 1
+    fi
   fi
+
   if [[ "${newResultsRaw}" == *"API rate limit exceeded"* ]]; then
-    echo "GitHub API rate limit exceeded. Please set GITHUB_USERNAME and GITHUB_TOKEN to avoid this."
+    echo "GitHub API rate limit exceeded. Please set GITHUB_TOKEN to avoid this."
     exit 1
   fi
 

releasing/create-release.sh

@@ -0,0 +1,116 @@
+#!/bin/bash
+# Copyright 2023 The Kubernetes Authors.
+# SPDX-License-Identifier: Apache-2.0
+
+#
+# This script is called by Kustomize's release pipeline.
+# It needs jq (required for release note construction) and [GitHub CLI](https://cli.github.com/).
+#
+# To test it locally:
+#
+#   # Please install jq and GitHub CLI. (e.g. macOS)
+#   brew install jq gh
+#
+#   # Setup GitHub CLI
+#   gh auth login
+#
+#   # Run this script, where $TAG is the tag to "release" (e.g. kyaml/v0.13.4)
+#   ./releasing/create-release.sh $TAG
+#
+#   # Please remove Draft Release created by this script.
+
+set -o errexit
+set -o nounset
+set -o pipefail
+
+if [[ -z "${1-}" ]]; then
+  echo "Usage: $0 TAG"
+  echo "  TAG: the tag to build or release, e.g. api/v1.2.3"
+  exit 1
+fi
+
+git_tag=$1
+echo "release tag: $git_tag"
+
+# Build the release binaries for every OS/arch combination.
+# It builds compressed artifacts on $release_dir.
+function build_kustomize_binary {
+  echo "build kustomize binaries"
+  version=$1
+
+  release_dir=$2
+  echo "build release artifacts to $release_dir"
+
+  mkdir -p "output"
+  # build date in ISO8601 format
+  build_date=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
+  for os in linux darwin windows; do
+    arch_list=(amd64 arm64)
+    if [ "$os" == "linux" ]; then
+      arch_list=(amd64 arm64 s390x ppc64le)
+    fi
+    for arch in "${arch_list[@]}" ; do
+      echo "Building $os-$arch"
+    #   CGO_ENABLED=0 GOWORK=off GOOS=$os GOARCH=$arch go build -o output/kustomize -ldflags\
+      CGO_ENABLED=0 GOOS=$os GOARCH=$arch go build -o output/kustomize -ldflags\
+        "-s -w\
+        -X sigs.k8s.io/kustomize/api/provenance.version=$version\
+        -X sigs.k8s.io/kustomize/api/provenance.gitCommit=$(git rev-parse HEAD)\
+        -X sigs.k8s.io/kustomize/api/provenance.buildDate=$build_date"\
+        kustomize/main.go
+      if [ "$os" == "windows" ]; then
+        zip "${release_dir}/kustomize_${version}_${os}_${arch}.zip" output/kustomize
+      else
+        tar cvfz "${release_dir}/kustomize_${version}_${os}_${arch}.tar.gz" output/kustomize
+      fi
+      rm output/kustomize
+    done
+    rmdir output
+  done
+}
+
+function create_release {
+  git_tag=$1
+
+  # Take everything before the last slash.
+  # This is expected to match $module.
+  module=${git_tag%/*}
+
+  # Take everything after the last slash.
+  version=${git_tag##*/}
+
+  # Generate the changelog for this release
+  # using the last two tags for the module
+  changelog_file=$(mktemp)
+  ./releasing/compile-changelog.sh "$module" "$git_tag" "$changelog_file"
+
+  additional_release_artifacts_arg=""
+
+  # build `kustomize` binary
+  if [[ "$module" == "kustomize" ]]; then
+    release_artifact_dir=$(mktemp -d)
+    build_kustomize_binary "$version" "$release_artifact_dir"
+
+    # additional_release_artifacts_arg+="$release_artifact_dir/*"
+    additional_release_artifacts_arg=("$release_artifact_dir"/*)
+
+    # create github releases
+    gh release create "$git_tag" \
+      --title "$git_tag"\
+      --draft \
+      --notes-file "$changelog_file"\
+      "${additional_release_artifacts_arg[@]}"
+
+    return
+  fi
+
+  # create github releases
+  gh release create "$git_tag" \
+    --title "$git_tag"\
+    --draft \
+    --notes-file "$changelog_file"
+}
+
+
+## create release
+create_release "$git_tag"

releasing/run-goreleaser.sh

@@ -1,155 +0,0 @@
-#!/bin/bash
-# Copyright 2022 The Kubernetes Authors.
-# SPDX-License-Identifier: Apache-2.0
-
-#
-# Builds and optionally releases the specified module
-#
-# Usage (from top of repo):
-#
-#  releasing/run-goreleaser.sh TAG MODE[build|release] [--snapshot]
-#
-# Where TAG is in the form
-#
-#   api/v1.2.3
-#   kustomize/v1.2.3
-#   cmd/config/v1.2.3
-#   ... etc.
-#
-
-set -o errexit
-set -o nounset
-set -o pipefail
-
-if [[ -z "${1-}" || -z "${2-}" ]]; then
-  echo "Usage: $0 TAG MODE [goreleaser flags]"
-  echo "  TAG: the tag to build or release, e.g. api/v1.2.3"
-  echo "  MODE: build or release"
-  exit 1
-fi
-
-fullTag=$1
-shift
-echo "fullTag=$fullTag"
-export GORELEASER_CURRENT_TAG=$fullTag
-
-if [[ $1 == "release" || $1 == "build" ]]; then
-  mode=$1
-  shift
-else
-  echo "Error: mode must be build or release"
-  exit 1
-fi
-
-remainingArgs="$@"
-echo "Remaining args: $remainingArgs"
-
-# Take everything before the last slash.
-# This is expected to match $module.
-module=${fullTag%/*}
-echo "module=$module"
-
-# Take everything after the last slash.
-# This should be something like "v1.2.3".
-semVer=${fullTag#$module/}
-echo "semVer=$semVer"
-
-# Generate the changelog for this release
-# using the last two tags for the module
-changeLogFile=$(mktemp)
-./releasing/compile-changelog.sh "$module" "$fullTag" "$changeLogFile"
-echo
-echo "######### Release notes: ##########"
-cat "$changeLogFile"
-echo "###################################"
-echo
-
-# This is probably a directory called /workspace
-
-# Sanity check
-echo
-echo "############ DEBUG ##############"
-echo "pwd = $PWD"
-echo "ls -las ."
-ls -las .
-echo "###################################"
-echo
-
-# CD into the module directory.
-# This directory expected to contain a main.go, so there's
-# no need for extra details in the `build` stanza below.
-cd $module
-
-# This is used in goreleaser.yaml
-skipBuild=true
-if [[ "$module" == "kustomize" || "$module" == "pluginator" ]]; then
-  # If releasing a main program, don't skip the build.
-  skipBuild=false
-fi
-
-goReleaserConfigFile=$(mktemp)
-
-cat <<EOF >$goReleaserConfigFile
-project_name: $module
-
-archives:
-- name_template: "${module}_${semVer}_{{ .Os }}_{{ .Arch }}"
-
-builds:
-- skip: $skipBuild
-
-  ldflags: >
-    -s
-    -X sigs.k8s.io/kustomize/api/provenance.version={{.Version}}
-    -X sigs.k8s.io/kustomize/api/provenance.buildDate={{.Date}}
-
-  goos:
-  - linux
-  - darwin
-  - windows
-
-  goarch:
-  - amd64
-  - arm64
-  - s390x
-  - ppc64le
-
-checksum:
-  name_template: 'checksums.txt'
-
-env:
-- CGO_ENABLED=0
-- GO111MODULE=on
-- GOWORK=off
-
-release:
-  github:
-    owner: kubernetes-sigs
-    name: kustomize
-  draft: true
-
-EOF
-
-echo
-echo "############# CONFIG ##############"
-cat "$goReleaserConfigFile"
-echo "###################################"
-echo
-
-args=(
-  --debug
-  --timeout 10m
-  --parallelism 7
-  --config="$goReleaserConfigFile"
-  --rm-dist
-  --skip-validate
-)
-if [[ $mode == "release" ]]; then
-  args+=(--release-notes="$changeLogFile")
-fi
-
-date
-export PATH="/usr/local/bin:$PATH"
-set -x
-time goreleaser "$mode" "${args[@]}" $remainingArgs
-date