senaduka/old-web
1
0
Fork 0
Code Issues Pull Requests 5 Actions Packages Projects Releases Wiki Activity

add new ENV variables to control crawler

Browse Source
This commit is contained in:
Bilal Catic
2019-09-18 15:33:09 +02:00
parent 3d46c82d3d
commit d3b2f95719
1 changed files with 8 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
development.env
View File

@@ -13,3 +13,11 @@ AMAZON_SECRET_ACCESS_KEY=(your-key-here)
AMAZON_REGION=eu-west-1
APP_URL=http://localhost:3001
SOURCE_EMAIL=info@saburly.com
#=============== CRAWLER SETTINGS===============#
#==OLX==
OLX_START_PAGE=Crawler starts from this page
OLX_END_PAGE=Crawler ends with this page (including this page)
OLX_MAX_RESULTS_PER_PAGE=Only this number or less results from one page will be scraped and saved
OLX_CRAWLER_AD_TYPE=enum name of what type of ads should be crawled, check common/enums.js file for valid values
OLX_CRAWLER_AD_CATEGORIES=comma separated list of enum names of categories to be included, check common/enums.js file for valid values