kubernetes-sigs/kustomize
1
0
Fork 0
mirror of https://github.com/kubernetes-sigs/kustomize.git synced 2026-06-10 08:20:59 +00:00
Code Issues Packages Projects Releases 1 Wiki Activity

Add ElasticSearch query examples

Browse Source
This commit is contained in:
Haiyan Meng
2019-12-20 10:02:27 -08:00
parent 837df94d67
commit e2b56910f9
7 changed files with 625 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

180
api/internal/crawl/search_cmds/creationTime.md Normal file
View File

@@ -0,0 +1,180 @@
Find out the largest value of the `creationTime` field:
```
curl -X POST "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"aggs" : {
"max_creationTime" : { "max" : { "field" : "creationTime" } }
}
}
'
```
Find out the smallest value of the `creationTime` field:
```
curl -X POST "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"aggs" : {
"min_creationTime" : { "min" : { "field" : "creationTime" } }
}
}
'
```
Find out the smallest value of the `creationTime` field of all the kustomization files:
```
curl -X POST "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "filePath": ".*/kustomization((.yaml)?|(.yml)?)/*" }}
]
}
},
"aggs" : {
"min_creationTime" : { "min" : { "field" : "creationTime" } }
}
}
'
```
Find out the smallest value of the `creationTime` field of all kustomize resource files:
```
curl -X POST "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": ".*/kustomization((.yaml)?|(.yml)?)/*" }
}
}
},
"aggs" : {
"min_creationTime" : { "min" : { "field" : "creationTime" } }
}
}
'
```
Query all the documents whose `creationTime` <= `2016-07-29T17:38:26.000Z`:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"range": {
"creationTime": {
"lte": "2016-07-29T17:38:26.000Z"
}
}
}
}
'
```
Query all the documents whose `creationTime` falls within the specific range:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"range": {
"creationTime": {
"gte": "2016-07-29T17:38:26.000Z",
"lte": "2016-08-29T17:38:26.000Z"
}
}
}
}
'
```
Aggregate how many new kustomization files were added into Github each month:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "filePath": ".*/kustomization((.yaml)?|(.yml)?)/*" }}
]
}
},
"aggs" : {
"newFiles_over_time" : {
"date_histogram" : {
"field" : "creationTime",
"interval" : "month"
}
}
}
}
'
```
Aggregate how many new kustomize resource files were added into Github each month:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": [
{ "regexp": { "filePath": ".*/kustomization((.yaml)?|(.yml)?)/*" }}
]
}
},
"aggs" : {
"newFiles_over_time" : {
"date_histogram" : {
"field" : "creationTime",
"interval" : "month"
}
}
}
}
'
```
Aggregate how many new kustomization files were added into Github each year:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "filePath": ".*/kustomization((.yaml)?|(.yml)?)/*" }}
]
}
},
"aggs" : {
"newFiles_over_time" : {
"date_histogram" : {
"field" : "creationTime",
"interval" : "year"
}
}
}
}
'
```
Aggregate how many new kustomize resource files were added into Github each year:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": [
{ "regexp": { "filePath": ".*/kustomization((.yaml)?|(.yml)?)/*" }}
]
}
},
"aggs" : {
"newFiles_over_time" : {
"date_histogram" : {
"field" : "creationTime",
"interval" : "year"
}
}
}
}
'
```

32
api/internal/crawl/search_cmds/defaultBranch.md Normal file
View File

@@ -0,0 +1,32 @@
Count distinct values of the `defaultBranch` field:
```
curl -X POST "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"aggs" : {
"defaultBranch_count" : {
"cardinality" : {
"field" : "defaultBranch",
"precision_threshold": 40000
}
}
}
}
'
```
List all the github branches where kustomization files and kustomize resource files live,
and how many kustomization files and kustomize resource files live in each branch:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"aggs" : {
"defaultBranch" : {
"terms" : {
"field" : "defaultBranch",
"size": 41
}
}
}
}
'
```

55
api/internal/crawl/search_cmds/fieldExistence.md Normal file
View File

@@ -0,0 +1,55 @@
Count the documents whose `document` field is empty (The reason why the `document` field
of a document is empty is because of empty documents):
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"size": 10000,
"query": {
"bool": {
"must_not": {
"exists": {
"field": "document"
}
}
}
}
}
'
```
Find all the documents having the `creationTime` field set:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"exists": {
"field": "creationTime"
}
}
}
'
```
Find all the documents whose `creationTime` field is not set:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"size": 10000,
"query": {
"bool": {
"must_not": {
"exists": {
"field": "creationTime"
}
}
}
}
}
'
```
The following fields of a document in the kustomize index are always non-empty:
`repositoryUrl`, `filePath`, `defaultBranch`.
The following fields of a document in the kustomize index may be empty:
`kinds`, `identifiers`, `values`.

66
api/internal/crawl/search_cmds/keyword_search.md Normal file
View File

@@ -0,0 +1,66 @@
Count the documents in the index whose `repositoryUrl` field starts with
`https://github.com/`:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "repositoryUrl": "https://github.com/.*" }}
]
}
}
}
'
```
Count the documents in the index whose `repositoryUrl` field does not start with
`https://github.com/`:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": [
{ "regexp": { "repositoryUrl": "https://github.com/.*" }}
]
}
}
}
'
```
Search all the documents matching the given `repositoryUrl` and `filePath`, and return
a version for each search hit:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"size": 10000,
"version": true,
"query": {
"bool": {
"filter": [
{ "regexp": { "repositoryUrl": "git@github.com:talos-systems/talos-controller-manager" }},
{ "regexp": { "filePath": "hack/config.*" }}
]
}
}
}
'
```
Search all the documents whose filePath ends with one of these following three filenames:
`kustomization.yaml`, `kustomization.yml`, `kustomization`:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "filePath": ".*/kustomization((.yaml)?|(.yml)?)/*" }}
]
}
}
}
'
```

19
api/internal/crawl/search_cmds/misc.md Normal file
View File

@@ -0,0 +1,19 @@
Check the health status of an ElasticSearch cluster:
```
curl -X GET "${ElasticSearchURL}:9200/_cat/health?v&pretty"
```
Check the indices in an ElasticSearch cluster:
```
curl "${ElasticSearchURL}:9200/_cat/indices?v"
```
Get the mapping of the index:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_mapping?pretty"
```
Delete the kustomize index from the ElasticSearch cluster (**Use this command with caution**):
```
curl -X DELETE "${ElasticSearchURL}:9200/kustomize?pretty"
```

125
api/internal/crawl/search_cmds/repositoryUrl.md Normal file
View File

@@ -0,0 +1,125 @@
Count distinct values of the `repositoryUrl` field:
```
curl -X POST "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"aggs" : {
"repositoryUrl_count" : {
"cardinality" : {
"field" : "repositoryUrl",
"precision_threshold": 40000
}
}
}
}
'
```
Count how many Github repositories include kustomization files:
```
curl -X POST "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "filePath": ".*/kustomization((.yaml)?|(.yml)?)/*" }}
]
}
},
"aggs" : {
"repositoryUrl_count" : {
"cardinality" : {
"field" : "repositoryUrl",
"precision_threshold": 40000
}
}
}
}
'
```
Count how many Github repositories include kustomize resource files:
```
curl -X POST "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": ".*/kustomization((.yaml)?|(.yml)?)/*" }
}
}
},
"aggs" : {
"repositoryUrl_count" : {
"cardinality" : {
"field" : "repositoryUrl",
"precision_threshold": 40000
}
}
}
}
'
```
List all the github repositories including kustomization files and kustomize resource files,
and how many kustomization files and kustomize resource files each github repository includes
(the github repository including the most kustomization files is listed first):
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"aggs" : {
"repositoryUrl" : {
"terms" : {
"field" : "repositoryUrl",
"size": 2082
}
}
}
}
'
```
List the top 20 Github repositories including the most amount of kustomization files:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "filePath": ".*/kustomization((.yaml)?|(.yml)?)/*" }}
]
}
},
"aggs" : {
"repositoryUrl" : {
"terms" : {
"field" : "repositoryUrl",
"size": 20
}
}
}
}
'
```
List the top 20 Github repositories including the most amount of kustomize resource files:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?size=0&pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": {
"regexp": { "filePath": ".*/kustomization((.yaml)?|(.yml)?)/*" }
}
}
},
"aggs" : {
"repositoryUrl" : {
"terms" : {
"field" : "repositoryUrl",
"size": 20
}
}
}
}
'
```

148
api/internal/crawl/search_cmds/text_search.md Normal file
View File

@@ -0,0 +1,148 @@
Search for all the kustomize resource files including a Deployment object:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"match" : {
"kinds" : {
"query" : "Deployment"
}
}
}
}
'
```
Search for all the kustomize resource files including a Deployment object, but only
including the `kinds` field in the result:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"_source": {
"includes": ["kinds"]
},
"query": {
"match" : {
"kinds" : {
"query" : "Deployment"
}
}
}
}
'
```
Search for all the kustomize resource files including both a Deployment object and
a Service object:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"match" : {
"kinds" : {
"query" : "Deployment Service",
"operator" : "and"
}
}
}
}
'
```
Count the number of documents including Deployment and the number of documents
including Service:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"size": 0,
"aggs" : {
"messages" : {
"filters" : {
"filters" : {
"Deployment" : { "match" : { "kinds" : "Deployment" }},
"Service" : { "match" : { "kinds" : "Service" }}
}
}
}
}
}
'
```
Search for all the kustomization files involving CRDs:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"size": 10000,
"query": {
"match" : {
"identifiers" : {
"query" : "crds"
}
}
}
}
'
```
Search for all the kustomization files defining configMapGenerator:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"size": 10000,
"query": {
"match" : {
"identifiers" : {
"query" : "configMapGenerator"
}
}
}
}
'
```
Search for all the documents having a `kind` field:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "match" : { "identifiers" : { "query" : "kind" }}}
]
}
}
}
'
```
Search for all the kuostmization files having a `kind` field:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{ "regexp": { "filePath": ".*/kustomization((.yaml)?|(.yml)?)" }},
{ "match" : { "identifiers" : { "query" : "kind" }}}
]
}
}
}
'
```
Search for all the kustomization files defining the `generatorOptions:disableNameSuffixHash` feature:
```
curl -X GET "${ElasticSearchURL}:9200/kustomize/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"match" : {
"identifiers" : {
"query" : "generatorOptions:disableNameSuffixHash"
}
}
}
}
'
```