Commit Graph

15 Commits

Author SHA1 Message Date
=
bf8d131025 Increase timeout to 20 secs 2020-09-15 01:34:07 -07:00
=
698acb010a Add timeout to fetch wrapper 2020-09-15 01:27:20 -07:00
Senad Uka
e4775158fc Promise pool 2020-09-10 19:39:13 +02:00
Senad Uka
26377c485c Reverting 2020-09-10 18:30:14 +02:00
Senad Uka
b30b0f45a6 trying different approach 2020-09-10 18:23:01 +02:00
Senad Uka
9c1a029ff1 Debugging ba 2020-09-10 18:17:16 +02:00
Senad Uka
9b49759485 Pause is not enough 2020-09-10 18:13:47 +02:00
Senad Uka
0c2f8d11ee Add random timeout up to 500ms 2020-09-10 18:10:42 +02:00
Senad Uka
dd3f30ef0e Comment agent header 2020-09-10 17:57:21 +02:00
Senad Uka
78c6056db4 Change for scraperapi 2020-09-10 17:37:19 +02:00
Bilal
ff923605ad Move scraper api base url to the ENV; send URL as base64 string 2020-05-12 13:44:09 +02:00
Naida Vatric
ce857ddce9 Renamed var. 2020-02-23 23:11:21 +01:00
Naida Vatric
d436d4a37b Added Scraper API option. 2020-02-22 22:15:27 +01:00
Bilal Catic
6791a509d0 make user agent header configurable through env variable 2020-02-20 21:07:16 +01:00
Bilal Catic
f62a7200c7 create fetch wrapper with mandatory user agent header 2020-02-20 19:47:30 +01:00