iNocturnis/webcrawler
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

more todos

Browse Source
This commit is contained in:
Lacerum 2022-04-18 18:38:16 -07:00
parent 4080d46541
commit 4ace2164f2
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
spacetime-crawler4py-master/scraper.py
View File

@ -141,7 +141,7 @@ def is_valid(url):
# https://support.archive-it.org/hc/en-us/articles/208332963-Modify-crawl-scope-with-a-Regular-Expression # https://support.archive-it.org/hc/en-us/articles/208332963-Modify-crawl-scope-with-a-Regular-Expression
# add lem check # add lem check
# add another dir check # add another dir check
# add extra dir check # add extra dir check (we can add as we find)
# add cal check # add cal check
else: else:
return True return True