Releases: viperior/sea-spider
Releases · viperior/sea-spider
v1.3.1
v1.3.0
Issues fixed
- Ensure the log file is created automatically when the spider is activated for the first time
- Replace calls to the deprecated function, logging.warn() with logging.warning()
- Bump urllib3 version from 1.26.3 to 1.26.4
Improvements
- Add timestamp to log lines
- Add VS Code settings directory to .gitignore
- Add namespace condition to the self-call to main() in the seaspider module
v1.2.0
v1.1.0
New features
- Crawl a list of URLs stored in CSV format
- Generate HTML report with crawl results
- Quickly purge the crawl cache with
purge_cache.py
Issues fixed
- Classify
http:
links without redirects as problems
Improvements
- Add more detail on basic usage and modifying values in the config file