Files
kustomize/internal
Damien Robichaud e0d388c6f7 Implements search query partitioning by filesize.
Binary searches through different ranges of file sizes to create search
queries with fewer than 1000 results. This is required since github will
only return the first 1000 result of any search query.

The implementation handles the case where some files may be deleted
while the search is running, and (possibly artificially) assures that the
number of files increases monotonically as the filesize range grows.

The implementation also caches queries and is expected to make fewer
than O((#files/1000) * lg(max file size)) API calls to retrieve the
range queries that can be used to index all of the files.

In practice running the search splitting takes a few minutes, while
retrieving all of the data takes a few hours.
2019-08-16 09:48:23 -07:00
..