Use RWMutex to control the map access

Without RWMutex, we may run into fatal error: concurrent map read and map write.
This commit is contained in:
Haiyan Meng
2020-06-14 13:40:08 -07:00
parent a895220743
commit 171412cc98
2 changed files with 22 additions and 6 deletions

View File

@@ -191,7 +191,7 @@ func main() {
// this greatly reduces the time overhead of CrawlGithub.
getSeedDocsFunc()
for _, d := range seedDocs {
seen[d.ID()] = d.FileType
seen.Set(d.ID(), d.FileType)
}
crawler.CrawlGithub(ctx, crawlers, docConverter, indexFunc, seen)
case CrawlUser: