kubernetes-sigs/kustomize
1
0
Fork 0
mirror of https://github.com/kubernetes-sigs/kustomize.git synced 2026-05-21 22:41:42 +00:00
Code Issues Packages Projects Releases 1 Wiki Activity

Add curl commands for querying different filetypes

Browse Source
This commit is contained in:
Haiyan Meng
2020-01-23 16:04:55 -08:00
parent 0820865e1d
commit b7b88cae76
4 changed files with 781 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

211
api/internal/crawl/search_cmds/creationTime.md
View File

@@ -46,7 +46,52 @@ curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'C
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
}
},
"filter": [
{ "regexp": { "fileType": "resource" }}
]
}
},
"aggs" : {
"min_creationTime" : { "min" : { "field" : "creationTime" } }
}
}
'
```
Find out the smallest value of the `creationTime` field of all kustomize generator files:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"filter": [
{ "regexp": { "fileType": "generator" }}
]
}
},
"aggs" : {
"min_creationTime" : { "min" : { "field" : "creationTime" } }
}
}
'
```
Find out the smallest value of the `creationTime` field of all kustomize transformer files:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"filter": [
{ "regexp": { "fileType": "transformer" }}
]
}
},
"aggs" : {
@@ -142,6 +187,61 @@ curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Co
"bool": {
"must_not": [
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
],
"filter": [
{ "regexp": { "fileType": "resource" }}
]
}
},
"aggs" : {
"newFiles_over_time" : {
"date_histogram" : {
"field" : "creationTime",
"interval" : "month"
}
}
}
}
'
```
Aggregate how many new kustomize generator files were added into Github each month:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": [
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
],
"filter": [
{ "regexp": { "fileType": "generator" }}
]
}
},
"aggs" : {
"newFiles_over_time" : {
"date_histogram" : {
"field" : "creationTime",
"interval" : "month"
}
}
}
}
'
```
Aggregate how many new kustomize transformer files were added into Github each month:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": [
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
],
"filter": [
{ "regexp": { "fileType": "transformer" }}
]
}
},
@@ -188,7 +288,10 @@ curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Co
"bool": {
"must_not": [
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
]
],
"filter": [
{ "regexp": { "fileType": "resource" }}
]
}
},
"aggs" : {
@@ -201,4 +304,108 @@ curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Co
}
}
'
```
Aggregate how many new kustomize generator files were added into Github each year:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": [
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
],
"filter": [
{ "regexp": { "fileType": "generator" }}
]
}
},
"aggs" : {
"newFiles_over_time" : {
"date_histogram" : {
"field" : "creationTime",
"interval" : "year"
}
}
}
}
'
```
Aggregate how many new kustomize transformer files were added into Github each year:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": [
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
],
"filter": [
{ "regexp": { "fileType": "transformer" }}
]
}
},
"aggs" : {
"newFiles_over_time" : {
"date_histogram" : {
"field" : "creationTime",
"interval" : "year"
}
}
}
}
'
```
Find the generator files created within the given time range:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "generator" }}
],
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"must": {
"range": {
"creationTime": {
"gte": "2019-04-26T16:40:02.000Z",
"lte": "2019-04-26T16:40:02.000Z"
}
}
}
}
}
}
'
```
Find the transformer files created within the given time range:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "transformer" }}
],
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"must": {
"range": {
"creationTime": {
"gte": "2019-04-26T16:40:02.000Z",
"lte": "2019-04-26T16:40:02.000Z"
}
}
}
}
}
}
'
```

178
api/internal/crawl/search_cmds/fileType.md
View File

@@ -44,6 +44,184 @@ curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-T
'
```
Search for all the kustomization files whose `fileType` field is `resource`:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }},
{ "regexp": { "fileType": "resource" }}
]
}
}
}
'
```
Search for all the kustomize resource files:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "resource" }}
],
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
}
}
}
}
'
```
Search all the kustomization files including a `generators` field:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-Type: application/json' -d'
{
"size": 10000,
"query": {
"bool": {
"must": {
"match" : {
"identifiers" : {
"query" : "generators"
}
}
},
"filter": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
}
}
}
}
'
```
Search for all the documents whose `fileType` field is `generator`:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "generator" }}
]
}
}
}
'
```
Search for all the kustomization files whose `fileType` field is `generator`:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }},
{ "regexp": { "fileType": "generator" }}
]
}
}
}
'
```
Search for all the kustomize generator files:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "generator" }}
],
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
}
}
}
}
'
```
Search all the kustomization files including a `transformers` field:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-Type: application/json' -d'
{
"size": 10000,
"query": {
"bool": {
"must": {
"match" : {
"identifiers" : {
"query" : "transformers"
}
}
},
"filter": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
}
}
}
}
'
```
Search for all the documents whose `fileType` field is `transformer`:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "transformer" }}
]
}
}
}
'
```
Search for all the kustomization files whose `fileType` field is `transformer`:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }},
{ "regexp": { "fileType": "transformer" }}
]
}
}
}
'
```
Search for all the kustomize transformer files:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "transformer" }}
],
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
}
}
}
}
'
```
Count distinct values of the `fileType` field:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'

136
api/internal/crawl/search_cmds/repositoryUrl.md
View File

@@ -37,16 +37,143 @@ curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'C
'
```
Count how many Github repositories include kustomize resource files:
Count distinct values of the `repositoryUrl` field for all the kustomize resource files in the index:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"filter": [
{ "regexp": { "fileType": "resource" }}
]
}
},
"aggs" : {
"repositoryUrl_count" : {
"cardinality" : {
"field" : "repositoryUrl",
"precision_threshold": 40000
}
}
}
}
'
```
Count distinct values of the `repositoryUrl` field for all the kustomize generator files in the index:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"filter": [
{ "regexp": { "fileType": "generator" }}
]
}
},
"aggs" : {
"repositoryUrl_count" : {
"cardinality" : {
"field" : "repositoryUrl",
"precision_threshold": 40000
}
}
}
}
'
```
Count distinct values of the `repositoryUrl` field for all the kustomize transformer files in the index:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"filter": [
{ "regexp": { "fileType": "transformer" }}
]
}
},
"aggs" : {
"repositoryUrl_count" : {
"cardinality" : {
"field" : "repositoryUrl",
"precision_threshold": 40000
}
}
}
}
'
```
Count distinct values of the `repositoryUrl` field for all the kustomize resource dirs in the index:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "resource" }},
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
]
}
},
"aggs" : {
"repositoryUrl_count" : {
"cardinality" : {
"field" : "repositoryUrl",
"precision_threshold": 40000
}
}
}
}
'
```
Count distinct values of the `repositoryUrl` field for all the kustomize generator dirs in the index:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "generator" }},
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
]
}
},
"aggs" : {
"repositoryUrl_count" : {
"cardinality" : {
"field" : "repositoryUrl",
"precision_threshold": 40000
}
}
}
}
'
```
Count distinct values of the `repositoryUrl` field for all the kustomize transformer dirs in the index:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "transformer" }},
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
]
}
},
"aggs" : {
"repositoryUrl_count" : {
@@ -109,7 +236,10 @@ curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Co
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
}
},
"filter": [
{ "regexp": { "fileType": "resource" }}
]
}
},
"aggs" : {

265
api/internal/crawl/search_cmds/user.md
View File

@@ -67,7 +67,8 @@ curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Co
"aggs" : {
"user" : {
"terms" : {
"field" : "user"
"field" : "user",
"size" : 20
}
}
}
@@ -75,6 +76,28 @@ curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Co
'
```
Count distinct values of the `user` field for all the kustomization files in the index:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
]
}
},
"aggs" : {
"user_count" : {
"cardinality" : {
"field" : "user",
"precision_threshold": 40000
}
}
}
}
'
```
For all the kustomization files in the index, list all the values of the
`user` field and the frequency of each value:
@@ -91,7 +114,8 @@ curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Co
"aggs" : {
"user" : {
"terms" : {
"field" : "user"
"field" : "user",
"size": 20
}
}
}
@@ -99,7 +123,33 @@ curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Co
'
```
For all the non-kustomization files in the index, list all the values of the
Count distinct values of the `user` field for all the kustomize resource files in the index:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"filter": [
{ "regexp": { "fileType": "resource" }}
]
}
},
"aggs" : {
"user_count" : {
"cardinality" : {
"field" : "user",
"precision_threshold": 40000
}
}
}
}
'
```
For all the kustomize resource files in the index, list all the values of the
`user` field and the frequency of each value:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
@@ -108,8 +158,215 @@ curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Co
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"filter": [
{ "regexp": { "fileType": "resource" }}
]
}
},
"aggs" : {
"user" : {
"terms" : {
"field" : "user",
"size": 20
}
}
}
}
'
```
Count distinct values of the `user` field for all the kustomize generator files in the index:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"filter": [
{ "regexp": { "fileType": "generator" }}
]
}
},
"aggs" : {
"user_count" : {
"cardinality" : {
"field" : "user",
"precision_threshold": 40000
}
}
}
}
'
```
For all the kustomize generator files in the index, list all the values of the
`user` field and the frequency of each value:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"filter": [
{ "regexp": { "fileType": "generator" }}
]
}
},
"aggs" : {
"user" : {
"terms" : {
"field" : "user",
"size": 20
}
}
}
}
'
```
Count distinct values of the `user` field for all the kustomize transformer files in the index:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"filter": [
{ "regexp": { "fileType": "transformer" }}
]
}
},
"aggs" : {
"user_count" : {
"cardinality" : {
"field" : "user",
"precision_threshold": 40000
}
}
}
}
'
```
For all the kustomize transformer files in the index, list all the values of the
`user` field and the frequency of each value:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }
},
"filter": [
{ "regexp": { "fileType": "transformer" }}
]
}
},
"aggs" : {
"user" : {
"terms" : {
"field" : "user",
"size": 20
}
}
}
}
'
```
Count distinct values of the `user` field for all the kustomize generator dirs in the index:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "generator" }},
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
]
}
},
"aggs" : {
"user_count" : {
"cardinality" : {
"field" : "user",
"precision_threshold": 40000
}
}
}
}
'
```
For all the kustomize generator dirs in the index, list all the values of the
`user` field and the frequency of each value:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "generator" }},
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
]
}
},
"aggs" : {
"user" : {
"terms" : {
"field" : "user",
"size": 20
}
}
}
}
'
```
Count distinct values of the `user` field for all the kustomize transformer dirs in the index:
```
curl -X POST "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "transformer" }},
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
]
}
},
"aggs" : {
"user_count" : {
"cardinality" : {
"field" : "user",
"precision_threshold": 40000
}
}
}
}
'
```
For all the kustomize transformer dirs in the index, list all the values of the
`user` field and the frequency of each value:
```
curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "fileType": "transformer" }},
{ "regexp": { "filePath": "(.*/)?kustomization((.yaml)?|(.yml)?)(/)*" }}
]
}
},
"aggs" : {
"user" : {
@@ -120,4 +377,4 @@ curl -X GET "${ElasticSearchURL}:9200/${INDEXNAME}/_search?size=0&pretty" -H 'Co
}
}
'
```
```