Jeff Regan
0ce076758d
Merge pull request #2150 from haiyanmeng/stats
...
Add `fileType` and `User` into the index
2020-01-28 09:18:31 -08:00
HowJMay
00f68c12a8
fix typos
...
Fix typos
2020-01-23 23:35:38 +08:00
Haiyan Meng
1120c6bc7a
Add a User field into Document to make it easy to aggregate on github
...
user level.
2020-01-21 10:09:52 -08:00
Haiyan Meng
f4636f8555
Add a fileType field into the index
2020-01-17 13:15:49 -08:00
Haiyan Meng
cf8d53a195
Move SeenMap to the utils dir
2020-01-15 15:29:16 -08:00
Haiyan Meng
2e895c147e
Use log.Print* instead of fmt.Print*
2020-01-14 15:50:35 -08:00
Haiyan Meng
7ac573ae51
Add a flag to specify the index name
2020-01-14 14:25:29 -08:00
Haiyan Meng
c801958d40
Log response status code to help debug
...
Recently, the crawler job often fails after 10+ hours with the following
error (10.0.47.27:9200 is the ElasticSearch master):
dial tcp 10.0.47.27:9200: connect: connection refused
2020-01-10 11:37:22 -08:00
Haiyan Meng
745b58b3d0
Check whether a pointer is empty before accessing it to avoid SIGSEGV
2020-01-06 12:06:18 -08:00
Haiyan Meng
bef157d6b3
Fix insert/updating document logic
2019-12-18 15:56:44 -08:00
Haiyan Meng
2c2aa928cc
Delete non-existing documents from the index
2019-12-18 15:56:44 -08:00
Haiyan Meng
272b7a6fcd
Use UpdateRequest to insert/update a document
...
Currently, `IndexRequest` is used to insert/update a document, which
increases the version of the document every time IndexRequest.Do is
called.
2019-12-18 15:56:44 -08:00
Haiyan Meng
8aaa3f56f5
Set the ElasticSearch index creation configuration
...
Currently, the `kustomize` index in ElasticSearch is using dynamic
mapping, which sets the types of all the fields to `text`. However,
`text` field type is good for full-text value matching, and not good for
exact-value matching. For exact-value matching, the `keyword` filed
type should be used.
2019-12-03 15:16:32 -08:00
Haiyan Meng
9255c991f4
Replace the sigs.k8s.io/kustomize/hack/crawl/* import path with
...
`sigs.k8s.io/kustomize/api/internal/crawl/*`
2019-11-14 13:38:18 -08:00
Haiyan Meng
f69d2d2e69
Move hack/crawl under api/internal
2019-11-14 13:17:28 -08:00
