From 46dbe4089118a80dc3d623d76181f3fdc939ac0c Mon Sep 17 00:00:00 2001 From: Bilal Catic Date: Sat, 2 Nov 2019 00:07:01 +0100 Subject: [PATCH] add missing ENV variables to env file template --- development.env | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/development.env b/development.env index f80dfe1..901c309 100644 --- a/development.env +++ b/development.env @@ -31,6 +31,7 @@ OLX_CRAWLER_AD_TYPE=enum name of what type of ads should be crawled, check commo OLX_CRAWLER_AD_CATEGORIES=comma separated list of enum names of categories to be included, check common/enums.js file for valid values OLX_IGNORED_USERNAMES=comma separated list of usernames to ignore OLX_DELAY_BETWEEN_PAGES=time in miliseconds to wait before indexing next page +OLX_FORCE_CRAWL=Non-zero value will force crawler to crawl all pages without stopping when known real estate is found #==RENTAL== RENTAL_MAX_PAGES=Restrict crawler to this number of pages RENTAL_MAX_RESULTS_PER_PAGE=Only this number or less results from one page will be scraped and saved @@ -38,6 +39,7 @@ RENTAL_CRAWLER_AD_TYPE=enum name of what type of ads should be crawled, check co RENTAL_CRAWLER_AD_CATEGORIES=comma separated list of enum names of categories to be included, check common/enums.js file for valid values RENTAL_IGNORED_USERNAMES=!!! This is not used for rental crawler !!! RENTAL_DELAY_BETWEEN_PAGES=time in miliseconds to wait before indexing next page +RENTAL_FORCE_CRAWL=Non-zero value will force crawler to crawl all pages without stopping when known real estate is found #==PROSTOR== PROSTOR_MAX_PAGES=!!! This is not used for prostor crawler !!! PROSTOR_MAX_RESULTS_PER_PAGE=For Prostor crawler, this represents MAX RESULTS in total @@ -45,6 +47,7 @@ PROSTOR_CRAWLER_AD_TYPE=enum name of what type of ads should be crawled, check c PROSTOR_CRAWLER_AD_CATEGORIES=comma separated list of enum names of categories to be included, check common/enums.js file for valid values PROSTOR_IGNORED_USERNAMES=!!! This is not used for prostor crawler !!! PROSTOR_DELAY_BETWEEN_PAGES=!!! This is not used for prostor crawler !!! +PROSTOR_FORCE_CRAWL=Non-zero value will force crawler to crawl all pages without stopping when known real estate is found #==AKTIDO== AKTIDO_MAX_PAGES=Restrict crawler to this number of pages AKTIDO_MAX_RESULTS_PER_PAGE=Only this number or less results from one page will be scraped and saved @@ -52,3 +55,4 @@ AKTIDO_CRAWLER_AD_TYPE=enum name of what type of ads should be crawled, check co AKTIDO_CRAWLER_AD_CATEGORIES=comma separated list of enum names of categories to be included, check common/enums.js file for valid values AKTIDO_IGNORED_USERNAMES=!!! This is not used for aktido crawler !!! AKTIDO_DELAY_BETWEEN_PAGES=time in miliseconds to wait before indexing next page +AKTIDO_FORCE_CRAWL=Non-zero value will force crawler to crawl all pages without stopping when known real estate is found