3 Commits

Author	SHA1	Message	Date
Bilal Catic	06d35fcb4b	move ignored usernames config to crawler specific config	2019-09-25 08:55:00 +02:00
Bilal Catic	3140fdf0c0	use function generator to index pages; crawl in parallel	2019-09-25 08:55:00 +02:00
Bilal Catic	3d46c82d3d	create new crawler and Postgres saver	2019-09-18 15:32:48 +02:00