Bilal Catic
|
88e7cac420
|
allow olx crawler to recognize other OLX categories
|
2019-10-14 09:27:32 +02:00 |
|
Bilal Catic
|
9fc5072632
|
include other olx real estate categories in enums and configs
|
2019-10-14 09:27:32 +02:00 |
|
Bilal Catic
|
0818fcecd2
|
remove crawler and saver logging
|
2019-10-10 00:59:12 +02:00 |
|
Bilal Catic
|
5e8e13a984
|
fix enums
|
2019-09-30 14:27:01 +02:00 |
|
Bilal Catic
|
9c0104a57c
|
refactor crawler - adapt to use new ENUM objects
|
2019-09-30 10:27:12 +02:00 |
|
Bilal Catic
|
3d203df988
|
remove comment from delay between indexing pages
|
2019-09-25 10:00:42 +00:00 |
|
Bilal Catic
|
c9a959f8be
|
stop crawling when existing, not renewed ad is found
|
2019-09-25 08:55:00 +02:00 |
|
Bilal Catic
|
90bc57edb6
|
stop crawling when existing, non-renewed ad is found
|
2019-09-25 08:55:00 +02:00 |
|
Bilal Catic
|
63eb64b0f6
|
parse and save published and renewed dates
|
2019-09-25 08:55:00 +02:00 |
|
Bilal Catic
|
3140fdf0c0
|
use function generator to index pages; crawl in parallel
|
2019-09-25 08:55:00 +02:00 |
|
Bilal Catic
|
c4f6c6e1c3
|
construct crawling url before indexing single page
|
2019-09-25 08:55:00 +02:00 |
|
Bilal Catic
|
3d46c82d3d
|
create new crawler and Postgres saver
|
2019-09-18 15:32:48 +02:00 |
|
Bilal Catic
|
76a989fa37
|
replace old crawler, without specific crawler and saver implementation
|
2019-09-16 15:59:53 +02:00 |
|