Commit Graph

471 Commits

Author SHA1 Message Date
Carl Henrik Lunde
5c7f8b8d73 perf: improve applyOrdering by avoid call to GetByCurrentId
This shaves of 14 seconds (one third) of the execution time for a
kustomization tree with 4000 documents, from 40.68s to 27.41s

            0     0%  5.44%     18.42s 40.56%  sigs.k8s.io/kustomize/api/krusty.(*Kustomizer).applySortOrder
            0     0%  5.44%     18.40s 40.52%  sigs.k8s.io/kustomize/api/internal/builtins.applyOrdering
before

    (pprof) top20 -cum
    Showing nodes accounting for 5.85s, 12.88% of 45.41s total
    Dropped 622 nodes (cum <= 0.23s)
    Showing top 20 nodes out of 157
        flat  flat%   sum%        cum   cum%
            0     0%     0%     40.68s 89.58%  github.com/spf13/cobra.(*Command).Execute
            0     0%     0%     40.68s 89.58%  github.com/spf13/cobra.(*Command).ExecuteC
            0     0%     0%     40.68s 89.58%  github.com/spf13/cobra.(*Command).execute
            0     0%     0%     40.68s 89.58%  main.main
            0     0%     0%     40.68s 89.58%  runtime.main
            0     0%     0%     40.68s 89.58%  sigs.k8s.io/kustomize/kustomize/v5/commands/build.NewCmdBuild.func1
            0     0%     0%     40.12s 88.35%  sigs.k8s.io/kustomize/api/krusty.(*Kustomizer).Run
        0.51s  1.12%  1.12%     33.20s 73.11%  sigs.k8s.io/kustomize/api/resource.(*Resource).CurId
            0     0%  1.12%     26.95s 59.35%  sigs.k8s.io/kustomize/api/resmap.(*resWrangler).GetMatchingResourcesByCurrentId
        0.35s  0.77%  1.89%     26.95s 59.35%  sigs.k8s.io/kustomize/api/resmap.(*resWrangler).filteredById
        0.07s  0.15%  2.05%     25.53s 56.22%  sigs.k8s.io/kustomize/api/resmap.GetCurrentId
            0     0%  2.05%     21.68s 47.74%  sigs.k8s.io/kustomize/api/internal/target.(*KustTarget).MakeCustomizedResMap (inline)
            0     0%  2.05%     21.68s 47.74%  sigs.k8s.io/kustomize/api/internal/target.(*KustTarget).makeCustomizedResMap
        0.54s  1.19%  3.24%     19.75s 43.49%  sigs.k8s.io/kustomize/api/resource.(*Resource).GetGvk (inline)
            1s  2.20%  5.44%     19.21s 42.30%  sigs.k8s.io/kustomize/kyaml/resid.GvkFromNode
            0     0%  5.44%     18.42s 40.56%  sigs.k8s.io/kustomize/api/internal/builtins.(*SortOrderTransformerPlugin).Transform
            0     0%  5.44%     18.42s 40.56%  sigs.k8s.io/kustomize/api/krusty.(*Kustomizer).applySortOrder
            0     0%  5.44%     18.40s 40.52%  sigs.k8s.io/kustomize/api/internal/builtins.applyOrdering
        0.87s  1.92%  7.36%     16.55s 36.45%  sigs.k8s.io/kustomize/kyaml/yaml.visitMappingNodeFields
        2.51s  5.53% 12.88%     15.68s 34.53%  sigs.k8s.io/kustomize/kyaml/yaml.visitFieldsWhileTrue

after

    (pprof) top20 -cum
    Showing nodes accounting for 1.23s, 3.85% of 31.98s total
    Dropped 584 nodes (cum <= 0.16s)
    Showing top 20 nodes out of 184
        flat  flat%   sum%        cum   cum%
            0     0%     0%     27.41s 85.71%  github.com/spf13/cobra.(*Command).Execute
            0     0%     0%     27.41s 85.71%  github.com/spf13/cobra.(*Command).ExecuteC
            0     0%     0%     27.41s 85.71%  github.com/spf13/cobra.(*Command).execute
            0     0%     0%     27.41s 85.71%  main.main
            0     0%     0%     27.41s 85.71%  runtime.main
            0     0%     0%     27.41s 85.71%  sigs.k8s.io/kustomize/kustomize/v5/commands/build.NewCmdBuild.func1
            0     0%     0%     26.85s 83.96%  sigs.k8s.io/kustomize/api/krusty.(*Kustomizer).Run
            0     0%     0%     22.07s 69.01%  sigs.k8s.io/kustomize/api/internal/target.(*KustTarget).MakeCustomizedResMap (inline)
            0     0%     0%     22.07s 69.01%  sigs.k8s.io/kustomize/api/internal/target.(*KustTarget).makeCustomizedResMap
        0.38s  1.19%  1.19%     20.69s 64.70%  sigs.k8s.io/kustomize/api/resource.(*Resource).CurId
            0     0%  1.19%     13.64s 42.65%  sigs.k8s.io/kustomize/api/resmap.(*resWrangler).Append
            0     0%  1.19%     13.55s 42.37%  sigs.k8s.io/kustomize/api/resmap.(*resWrangler).GetMatchingResourcesByCurrentId (inline)
        0.12s  0.38%  1.56%     13.55s 42.37%  sigs.k8s.io/kustomize/api/resmap.(*resWrangler).filteredById
        0.01s 0.031%  1.59%     12.67s 39.62%  sigs.k8s.io/kustomize/api/resmap.GetCurrentId
        0.21s  0.66%  2.25%     12.49s 39.06%  sigs.k8s.io/kustomize/api/resource.(*Resource).GetGvk (inline)
        0.51s  1.59%  3.85%     12.28s 38.40%  sigs.k8s.io/kustomize/kyaml/resid.GvkFromNode
            0     0%  3.85%     11.52s 36.02%  sigs.k8s.io/kustomize/api/internal/target.(*KustTarget).IgnoreLocal
            0     0%  3.85%     10.53s 32.93%  sigs.k8s.io/kustomize/api/internal/target.(*KustTarget).AccumulateTarget
            0     0%  3.85%     10.53s 32.93%  sigs.k8s.io/kustomize/api/internal/target.(*KustTarget).accumulateResources
            0     0%  3.85%     10.53s 32.93%  sigs.k8s.io/kustomize/api/internal/target.(*KustTarget).accumulateTarget
2023-10-30 22:31:52 +01:00
Natasha Sarkar
96ad106ee1 Use upstream go-yaml fork and remove our internal one (#5412)
* test new yaml fork

* delete old kyaml fork

* update to sigs yaml 1.4.0

* remove comments
2023-10-25 19:48:22 +02:00
koba1t
0465637335 Back to development mode; unpin the modules 2023-10-20 05:30:43 +09:00
koba1t
16395012d1 Update api to v0.15.0 2023-10-20 02:36:50 +09:00
koba1t
8668b74622 Update kyaml to v0.15.0 2023-10-20 01:07:49 +09:00
Jan-Otto Kröpke
d64ac13447 kustomize: Add support for OCI based helm repos (#5167)
* kustomize: Add support for OCI based helm repos

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* Update api/internal/builtins/HelmChartInflationGenerator.go

* Update plugin/builtin/helmchartinflationgenerator/HelmChartInflationGenerator.go

* go fmt

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* s/expectedHelmExternalDns/expectedHelmExternalDNS

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* commit

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* commit with content

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* Apply suggestions from maintainer

* go work sync

* added test

---------

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>
2023-10-17 21:24:00 +02:00
Kazuki Suda
f5ca753377 Update imdario/mergo to v0.3.13
Co-authored-by: Krzysztof Gibuła <krzysztof.gibula@gmail.com>
Signed-off-by: Kazuki Suda <kazuki.suda@gmail.com>
2023-09-24 16:34:45 +09:00
Kazuki Suda
0d4849ff98 Add a test case for issue 4905
```
===== ACTUAL BEGIN ========================================
apiVersion: v1
data:
  config.yaml: "null"
kind: ConfigMap
metadata:
  name: issue4905
===== ACTUAL END ==========================================
   EXPECTED              ACTUAL
   --------              ------
   apiVersion: v1        apiVersion: v1
   data:                 data:
X    config.yaml: |-       config.yaml: "null"
X      item1: 1          kind: ConfigMap
X      item2: 2          metadata:
X  kind: ConfigMap         name: issue4905
X  metadata:
X    name: issue4905
    hasgett.go:22: Expected not equal to actual
--- FAIL: TestHelmChartInflationGeneratorIssue4905 (0.24s)
```

Signed-off-by: Kazuki Suda <kazuki.suda@gmail.com>
2023-09-24 16:33:43 +09:00
yugo kobayashi
59696d1ace fix a patch files accept multiple patches (#5194)
* fix a patch files accept multiple patches

* fix comments and variable name

* add error handling when using not allowed multiple strategic-merge patches

* fix error message of Multiple Strategic-Merge Patch file

* refactor transformStrategicMerge()

* add TODO comment and test for Multiple JSON patch Yaml documents are not allowed

* refactoring PatchTransformer

* add multiple patch test for PatchTransformer package

* improve error message to PatchTransformer

* refactor const and error message check

* fix some error messages
2023-09-15 16:20:13 -07:00
Stephen Kitt
b692e49b1e Switch to json-patch v5
json-patch was bumped to v5 in k/k as a result of
https://github.com/kubernetes/kubernetes/pull/118384; this aligns with
that, but switches to the v5 module, which is documented as
gopkg.in/evanphx/json-patch.v5.

Signed-off-by: Stephen Kitt <skitt@redhat.com>
2023-08-26 08:42:07 +02:00
natasha41575
ff75dd6cd5 release cleanup 2023-07-31 12:27:44 -05:00
natasha41575
5ce14e5024 pin to cmd/config and api 2023-07-31 11:28:25 -05:00
natasha41575
4a893ce8c6 pin to kyaml 2023-07-31 11:08:00 -05:00
natasha41575
eeff67d88d unpin everything 2023-06-19 12:05:59 -05:00
natasha41575
5e7cc3437d pintoapi 2023-06-19 11:35:33 -05:00
natasha41575
8c0e0b3b47 pin to kyaml 2023-06-19 10:45:18 -05:00
koba1t
d1647a51af fix e2econtainer test 2023-06-07 02:19:55 +09:00
koba1t
5b76aa9d2f update go 1.20 2023-06-06 22:39:06 +09:00
Jefftree
7c98e531f2 make test-go-mod 2023-05-31 19:54:13 +00:00
natasha41575
fd508f0e3a release cleanup 2023-05-17 10:41:37 -05:00
natasha41575
84642b1fed pin to api v0.13.4 2023-05-09 14:48:02 -05:00
natasha41575
4b807107a6 pin to kyaml and cmd/config released versions 2023-05-09 14:27:34 -05:00
Natasha Sarkar
3d6f40bd5e Revert "Fix using same helm chart with different versions (#4999)"
This reverts commit 0f244a4a07.
2023-05-09 13:16:22 -05:00
natasha41575
529a25d30b unpin everything 2023-05-05 15:38:19 -05:00
natasha41575
7dcb2a50ff pin to api 0.13.3 2023-05-05 15:12:27 -05:00
natasha41575
1f98338481 pin to kyaml v0.14.2 2023-05-05 14:32:02 -05:00
Dustin Lish
0f244a4a07 Fix using same helm chart with different versions (#4999)
* Fix using same helm chart with different versions

* Fix p.ValuesFile when version is set

* Updated: Fix using same helm chart with different versions

* Add test for issue #4813

* Use if/else for readability, add version check to absChartHome
2023-04-24 15:09:02 -07:00
Yutaro
42bf3c0e2b prevent all uses of YAML aliases from being overwritten by a transformer (#5096)
* return copied Node

* add a test case about imageTagTransformer for anchor scenario

* add TestPatchTransformerAnchor

* TestReplacementTransformerAnchor
2023-03-31 16:07:50 -07:00
Kubernetes Prow Robot
5dff9df1f7 Merge pull request #5099 from justinsb/dont_swallow_helm_errors
Don't swallow helm execution errors
2023-03-20 12:35:09 -07:00
justinsb
c151147258 Don't swallow helm execution errors
These are otherwise hard to debug.
2023-03-20 19:14:07 +00:00
justinsb
317fcadccb Fix unused error value
The linter was complaining about err being unchecked.
2023-03-20 19:11:11 +00:00
Katrina Verey
0c92647760 Back to development mode; unpin the modules 2023-03-13 21:43:46 -04:00
Katrina Verey
fa90046136 Update api to v0.13.2 2023-03-13 21:07:40 -04:00
Katrina Verey
0c650423ed Update kyaml to v0.14.1 2023-03-13 19:24:20 -04:00
Katrina Verey
51e2714408 Back to development mode; unpin the modules 2023-02-07 19:56:39 -05:00
Katrina Verey
d825beff27 Update all modules to latest releases 2023-02-07 19:56:11 -05:00
Katrina Verey
e6c16a6ddc Restore previous test-go-mod and commit changes 2023-02-02 14:20:13 -05:00
Katrina Verey
03669a1804 Improve error message and fix typo in iampolicygenerator module name 2023-02-02 14:15:40 -05:00
Katrina Verey
ea8fc77f2f Update kyaml in the secondary modules 2023-02-01 16:55:56 -05:00
Natasha Sarkar
1957d5c746 support for more helm template args (#4926)
* support for more helm template args

* move templateArgs and unit tests to api/types

* undo package name change

* use our own simple helm chart instead of forking one

* add argument to AsHelmArgs

* code review

* lint errors
2023-02-01 10:19:05 -08:00
Anna Song
fb294921f5 Localize helm fields (#4978)
* Localize helm fields

* Address readability feedback

* Handle edge cases

* Improve readability

Split copyChartHome and use existence to prevent repeated copying.
2023-01-26 12:56:26 -08:00
Olivier Lemasle
aaf9d39975 Bump k8s.io/kube-openapi to remove archived dependency
Bump k8s.io/kube-openapi to remove archived dependency github.com/PuerkitoBio/urlesc
2023-01-13 20:50:59 +01:00
Cailyn
8ab4b8d167 remove direct dependency on github/pkg/errors (#4970) 2023-01-13 10:52:25 -08:00
Katrina Verey
18a60ef036 Uncommitted updated from running go mod tidy and go work sync 2023-01-13 12:53:11 -05:00
Kubernetes Prow Robot
445622147f Merge pull request #4889 from fengshunli/vuln
fix: all module net and text dependent package CVE-2022-41717 and CVE-2022-32149 vulnerabilities
2023-01-09 11:45:29 -08:00
fsl
32f31c34ba fix: all module net and text dependent package CVE-2022-41717 and CVE-2022-32149 vulnerabilities
Signed-off-by: fsl <1171313930@qq.com>
2023-01-09 11:27:35 +08:00
Katrina Verey
5be179305a Update sigs.k8s.io/yaml to 1.3.0 2023-01-06 16:23:17 -05:00
Khris Richardson
03c1534c38 HelmChartInflationGenerator: split on yaml document end markers 2022-12-06 14:35:43 -08:00
Yannis Zarkadas
a502717460 Make ordering configurable (#4019)
* api: Add new types for customizeable resource ordering

Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>

* plugins: Implement SortOrderTransformer plugin

Implement the SortOrderTransformer plugin. This plugin allows the user
to customize the order that kustomize will output resources in.

The API for the plugin is the following:

sortOptions:
  order: legacy | fifo
  legacySortOptions:
    orderFirst:
    - {GVK}
    orderLast:
    - {GVK}

Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>

* plugins: Add boilerplate and generate code for new SortOrderTransformer

Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>

* build: Add option to denote if the reorder flag was set by the user

We want to take different actions if the reorder flag was set by the
user or filled by the default value. Thus, we propagate this information
from build to the krusty options.

Signed-off-by: Yannis Zarkadas <yanniszark@gmail.com>

* api/krusty: Ensure sort ordering works with CLI flag and kustomization

Sort order can be defined in two places:
- (new) kustomization file
- (old) CLI flag
We want the kustomization file to take precedence over the CLI flag.

Eventually, we may want to move away from having a CLI flag altogether:
https://github.com/kubernetes-sigs/kustomize/issues/3947

Case 1: Sort order set in kustomization file AND in CLI flag.
Print a warning and let the kustomization file take precedence.

Case 2: Sort order set in CLI flag only or not at all.
Follow the CLI flag (defaults to legacy) and reorder at the end.

Case 3: Sort order set in kustomization file only.
Simply build the kustomization.

Signed-off-by: Yannis Zarkadas <yanniszark@gmail.com>

* krusty: Add e2e test for SortOrderTransformer

Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>

* plugins: Purge LegacyOrderTransformer

Signed-off-by: Yannis Zarkadas <yanniszark@gmail.com>

* Update go.work.sum

Signed-off-by: Yannis Zarkadas <yanniszark@gmail.com>

* review: Make review changes

Signed-off-by: Yannis Zarkadas <yanniszark@gmail.com>

Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>
Signed-off-by: Yannis Zarkadas <yanniszark@gmail.com>
2022-12-02 13:59:53 -08:00
Kubernetes Prow Robot
baf777c56e Merge pull request #4893 from KnVerey/pluginator-linter-bug
Fix pluginator incompatibility with nolint lines
2022-11-30 08:32:55 -08:00