kubernetes-sigs/kustomize
1
0
Fork 0
mirror of https://github.com/kubernetes-sigs/kustomize.git synced 2026-05-17 18:25:26 +00:00
Code Issues Packages Projects Releases 1 Wiki Activity

Add goplugin KV generator example.

Browse Source
This commit is contained in:
Jeffrey Regan
2019-03-26 17:49:11 -07:00
parent 99406d4412
commit 99391157ec
11 changed files with 319 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
examples/README.md
View File

@@ -28,14 +28,18 @@ go get sigs.k8s.io/kustomize
(e.g. devops/SRE and developers).
* [configGenerations](configGeneration.md) -
Rolling update when ConfigMapGenerator changes
Rolling update when ConfigMapGenerator changes.
* [secret generation](kvSourceGoPlugin.md) - Generating secrets.
* [generatorOptions](generatorOptions.md) - Modifying behavior of all ConfigMap and Secret generators.
* [generatorOptions](generatorOptions.md) -
Modifying behavior of all ConfigMap and Secret generators.
* [breakfast](breakfast.md) - Customize breakfast for
Alice and Bob.
* [vars](wordpress/README.md) - Injecting k8s runtime data into container arguments (e.g. to point wordpress to a SQL service) by vars.
* [vars](wordpress/README.md) - Injecting k8s runtime data into
container arguments (e.g. to point wordpress to a SQL service) by vars.
* [image names and tags](image.md) - Updating image names and tags without applying a patch.

279
examples/kvSourceGoPlugin.md Normal file
View File

@@ -0,0 +1,279 @@
[Secrets]: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#secret-v1-core
[ConfigMaps]: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#configmap-v1-core
[base64]: https://tools.ietf.org/html/rfc4648#section-4
[Go plugin]: https://golang.org/pkg/plugin
[v2.0.3]: https://github.com/kubernetes-sigs/kustomize/releases/tag/v2.0.3
# Generating Secrets
## What's a Secret?
Kubernetes [ConfigMaps] and [Secrets] are both
key:value (KV) maps, but the latter is intended to
signal that its values have a sensitive nature -
e.g. ssh keys or passwords.
Kubernetes assumes that the values in a Secret are
[base64] encoded, and decodes them before actual
use (as, say, the argument to a container
command). The user that creates the Secret must
base64 encode the data, or use a tool that does it
for them. This encoding doesn't protect the
secret from anything other than an
over-the-shoulder glance.
Protecting the actual secrecy of a Secret value is
up to the cluster operator. They must lock down
the cluster (and its `etcd` data store) as tightly
as desired, and likewise protect the bytes that
feed into the cluster to ultimately become the
content of a Secret value.
## Make a place to work
<!-- @establishBase @test -->
```
DEMO_HOME=$(mktemp -d)
```
## Secret values from local files
kustomize has three different ways to generate a secret
from local files:
* get them from so-called _env_ files (`NAME=VALUE`, one per line),
* consume the entire contents of a file to make one secret value,
* get literal values from the kustomization file itself.
Here's an example combining all three methods:
Make an env file with some short secrets:
<!-- @makeEnvFile @test -->
```
cat <<'EOF' >$DEMO_HOME/foo.env
ROUTER_PASSWORD=admin
DB_PASSWORD=iloveyou
EOF
```
Make a text file with a long secret:
<!-- @makeLongSecretFile @test -->
```
cat <<'EOF' >$DEMO_HOME/longsecret.txt
Lorem ipsum dolor sit amet,
consectetur adipiscing elit,
sed do eiusmod tempor incididunt
ut labore et dolore magna aliqua.
EOF
```
And make a kustomization file referring to the
above and additionally defining some literal KV
pairs:
<!-- @makeKustomization1 @test -->
```
cat <<'EOF' >$DEMO_HOME/kustomization.yaml
secretGenerator:
- name: mysecrets
kvSources:
- name: envfiles
pluginType: builtin
args:
- foo.env
- name: files
pluginType: builtin
args:
- longsecret.txt
- name: literals
pluginType: builtin
args:
- FRUIT=apple
- VEGETABLE=carrot
EOF
```
> The above syntax is _alpha_ behavior at HEAD, for v2.1+.
>
> The default value of `pluginType` is `builtin`, so the
> `pluginType` fields could be omitted.
>
> The equivalent [v2.0.3] syntax (still supported) is
> ```
> secretGenerator:
> - name: mysecrets
> env: foo.env
> files:
> - longsecret.txt
> literals:
> - FRUIT=apple
> - VEGETABLE=carrot
> ```
Now generate the Secret:
<!-- @build1 @test -->
```
result=$(kustomize build $DEMO_HOME)
echo "$result"
test 1 == $(echo "$result" | grep -c "FRUIT: YXBwbGU=")
```
This emits something like
> ```
> apiVersion: v1
> kind: Secret
> metadata:
> name: mysecrets-hfb5df789h
> type: Opaque
> data:
> FRUIT: YXBwbGU=
> VEGETABLE: Y2Fycm90
> ROUTER_PASSWORD: YWRtaW4=
> DB_PASSWORD: aWxvdmV5b3U=
> longsecret.txt: TG9yZW0gaXBzdW0gZG9sb3Igc2l0I... (elided)
> ```
The name of the resource is a prefix, `mysecrets`
(as specfied in the kustomization file), followed
by a hash of its contents.
Use your favorite base64 decoder to confirm the raw
versions of any of these values.
The problem that these three approaches share is
that the purported secrets must live on disk.
This adds additional security questions - who can
see the files, who installs them, who deletes
them, etc.
## Secret values from anywhere
> New _alpha_ behavior at HEAD, for v2.1+
A general alternative is to enshrine secret
value generation in a Go plugin.
The values can then come in via, say, an
authenticated and authorized RPC to a password
vault service.
Here's a trivial plugin that provides
hardcoded values:
<!-- @makePlugin @test -->
```
cat <<'EOF' >$DEMO_HOME/kvMaker.go
package main
var database = map[string]string{
"TREE": "oak",
"ROCKET": "Saturn V",
"FRUIT": "apple",
"VEGETABLE": "carrot",
"SIMPSON": "homer",
}
type plugin struct{}
var KVSource plugin
func (p plugin) Get(root string, args []string) (map[string]string, error) {
r := make(map[string]string)
for _, k := range args {
v, ok := database[k]
if ok {
r[k] = v
}
}
return r, nil
}
EOF
```
The two crucial items needed to
load and query the plugin are
1) the public symbol `KVSource`,
1) its public `Get` method and signature.
Plugins that generate KV pairs for kustomize
must be installed at
> ```
> $XDG_CONFIG_HOME/kustomize/plugins/kvSource
> ```
`XDG_CONFIG_HOME` is an environment variable
used by many programs as the root of a
configuration directory. If unspecified, the
default `$HOME/.config` is used. The rest of
the required directory path establishes that
the files found there are kustomize plugins
for generating KV pairs.
Compile and install the plugin:
<!-- @compilePlugin @test -->
```
kvSources=$DEMO_HOME/kustomize/plugins/kvSources
mkdir -p $kvSources
GOPATH=$DEMO_HOME:$GOPATH go build \
-buildmode plugin \
-o $kvSources/kvMaker.so \
$DEMO_HOME/kvMaker.go
```
Create a new kustomization file
referencing this plugin:
<!-- @makeKustomization2 @test -->
```
cat <<'EOF' >$DEMO_HOME/kustomization.yaml
secretGenerator:
- name: mysecrets
kvSources:
- name: kvMaker
pluginType: go
args:
- FRUIT
- VEGETABLE
EOF
```
Finally, generate the secret, setting
`XDG_CONFIG_HOME` appropriately:
<!-- @build2 @test -->
```
result=$(XDG_CONFIG_HOME=$DEMO_HOME kustomize \
--enable_alpha_goplugins_accept_panic_risk \
build $DEMO_HOME)
echo "$result"
test 1 == $(echo "$result" | grep -c "FRUIT: YXBwbGU=")
```
Specify the `--enable_...` flag to enable Go
plugins, which may fail if not compiled under
the same conditions as the main program. Try
this command without the flag to see more
explanation.
This should emit something like:
> ```
> apiVersion: v1
> kind: Secret
> metadata:
> name: mysecrets-bdt27dbkd6
> type: Opaque
> data:
> FRUIT: YXBwbGU=
> VEGETABLE: Y2Fycm90
> ```
i.e. a subset of the same values as above.

8
k8sdeps/configmapandsecret/basefactory.go
View File

@@ -91,7 +91,7 @@ func keyValuesFromLiteralSources(sources []string) ([]kv.Pair, error) {
}
func (bf baseFactory) keyValuesFromPlugins(sources []types.KVSource) ([]kv.Pair, error) {
var allKvs []kv.Pair
var result []kv.Pair
for _, s := range sources {
plug, err := bf.reg.Load(s.PluginType, s.Name)
if err != nil {
@@ -101,9 +101,11 @@ func (bf baseFactory) keyValuesFromPlugins(sources []types.KVSource) ([]kv.Pair,
if err != nil {
return nil, err
}
allKvs = append(allKvs, kvs...)
for k, v := range kvs {
result = append(result, kv.Pair{Key: k, Value: v})
}
}
return allKvs, nil
return result, nil
}
func (bf baseFactory) keyValuesFromFileSources(sources []string) ([]kv.Pair, error) {

12
k8sdeps/kv/plugin/builtin/envfiles.go
View File

@@ -21,16 +21,16 @@ import (
"sigs.k8s.io/kustomize/pkg/ifc"
)
// Envfiles format should be a path to a file to read lines of key=val
// EnvFiles format should be a path to a file to read lines of key=val
// pairs to create a configmap.
// i.e. a Docker .env file or a .ini file.
type Envfiles struct {
type EnvFiles struct {
Ldr ifc.Loader
}
// Get implements the interface for kv plugins.
func (p Envfiles) Get(root string, args []string) ([]kv.Pair, error) {
var all []kv.Pair
func (p EnvFiles) Get(root string, args []string) (map[string]string, error) {
all := make(map[string]string)
for _, path := range args {
if path == "" {
return nil, nil
@@ -43,7 +43,9 @@ func (p Envfiles) Get(root string, args []string) ([]kv.Pair, error) {
if err != nil {
return nil, err
}
all = append(all, kvs...)
for _, pair := range kvs {
all[pair.Key] = pair.Value
}
}
return all, nil
}

6
k8sdeps/kv/plugin/builtin/files.go
View File

@@ -32,8 +32,8 @@ type Files struct {
}
// Get implements the interface for kv plugins.
func (p Files) Get(root string, args []string) ([]kv.Pair, error) {
var kvs []kv.Pair
func (p Files) Get(root string, args []string) (map[string]string, error) {
kvs := make(map[string]string)
for _, s := range args {
k, fPath, err := kv.ParseFileSource(s)
if err != nil {
@@ -43,7 +43,7 @@ func (p Files) Get(root string, args []string) ([]kv.Pair, error) {
if err != nil {
return nil, err
}
kvs = append(kvs, kv.Pair{Key: k, Value: string(content)})
kvs[k] = string(content)
}
return kvs, nil
}

6
k8sdeps/kv/plugin/builtin/literals.go
View File

@@ -26,14 +26,14 @@ import (
type Literals struct{}
// Get implements the interface for kv plugins.
func (p Literals) Get(root string, args []string) ([]kv.Pair, error) {
var kvs []kv.Pair
func (p Literals) Get(root string, args []string) (map[string]string, error) {
kvs := make(map[string]string)
for _, s := range args {
k, v, err := kv.ParseLiteralSource(s)
if err != nil {
return nil, err
}
kvs = append(kvs, kv.Pair{Key: k, Value: v})
kvs[k] = v
}
return kvs, nil
}

2
k8sdeps/kv/plugin/builtinplugin.go
View File

@@ -34,7 +34,7 @@ func newBuiltinFactory(ldr ifc.Loader) *builtinFactory {
plugins: map[string]KVSource{
"literals": builtin.Literals{},
"files": builtin.Files{Ldr: ldr},
"envfiles": builtin.Envfiles{Ldr: ldr},
"envfiles": builtin.EnvFiles{Ldr: ldr},
},
}
}

23
k8sdeps/kv/plugin/goplugin.go
View File

@@ -29,23 +29,20 @@ var _ Factory = &goFactory{}
const (
kvSourcesDir = "kvSources"
EnableGoPluginsFlagName = "enable_alpha_goplugins_accept_panic_risk"
EnableGoPluginsFlagHelp = `
Warning: the main program may panic and exit on an
attempt to use a goplugin that was compiled under
conditions differing from the those in effect when
main was compiled. It's safest to use this flag in
the context of a container image holding both the
main and the goplugins it needs, all built on the
same machine, with the same transitive libs and
the same compiler version.
`
EnableGoPluginsFlagHelp = `The main program may panic and exit on an attempt
to use a goplugin that was compiled under conditions
differing from the those in effect when main was
compiled. It's safest to use this flag in the
context of a container image holding both the main
and the goplugins it needs, all built on the same
machine, with the same transitive libs and the same
compiler version.`
errorFmt = `
enable go plugins by specifying flag
--%s
Place .so files in
%s
%s
`
%s`
)
func newGoFactory(c *types.PluginConfig) *goFactory {
@@ -82,7 +79,7 @@ func (p *goFactory) load(name string) (KVSource, error) {
return nil, err
}
symbol, err := goPlugin.Lookup("Plugin")
symbol, err := goPlugin.Lookup("KVSource")
if err != nil {
return nil, err
}

6
k8sdeps/kv/plugin/plugin.go
View File

@@ -17,13 +17,9 @@ limitations under the License.
// Package plugin provides a plugin abstraction layer.
package plugin
import (
"sigs.k8s.io/kustomize/k8sdeps/kv"
)
// KVSource is the interface for kv source plugins.
type KVSource interface {
Get(root string, args []string) ([]kv.Pair, error)
Get(root string, args []string) (map[string]string, error)
}
// Factory is the interface for new kv source plugin implementations.

4
pkg/commands/commands.go
View File

@@ -55,12 +55,12 @@ See https://sigs.k8s.io/kustomize
PluginConfig: plugin.DefaultPluginConfig(),
}
c.Flags().BoolVar(
c.PersistentFlags().BoolVar(
&genMetaArgs.PluginConfig.GoEnabled,
plugin.EnableGoPluginsFlagName,
false, plugin.EnableGoPluginsFlagHelp)
// Not advertising this alpha feature.
c.Flags().MarkHidden(plugin.EnableGoPluginsFlagName)
c.PersistentFlags().MarkHidden(plugin.EnableGoPluginsFlagName)
uf := kunstruct.NewKunstructuredFactoryWithGeneratorArgs(&genMetaArgs)

1
pkg/pgmconfig/config.go
View File

@@ -23,6 +23,7 @@ import (
"runtime"
)
//noinspection GoSnakeCaseUsage
const (
XDG_CONFIG_HOME = "XDG_CONFIG_HOME"
defaultConfigSubdir = ".config"