Commit Graph

  • 70bd952ee1 Add ssl to sequelize master Senad Uka 2021-03-18 11:38:42 +01:00
  • 14039975c2 Modify time to live to 6 hours Senad Uka 2020-10-21 10:49:22 +02:00
  • 92e4f4ed5a Add caching to fetch wrapper Senad Uka 2020-10-21 06:16:35 +02:00
  • 88f9d10586 parseInt NODE_FETCH_TIMEOUT_MS = 2020-09-16 06:21:03 -07:00
  • 2b1cbcaa47 Cleanup after debugging = 2020-09-16 06:16:49 -07:00
  • bf8d131025 Increase timeout to 20 secs = 2020-09-15 01:34:07 -07:00
  • 698acb010a Add timeout to fetch wrapper = 2020-09-15 01:27:20 -07:00
  • ade3eb307d Add try/catch to generator = 2020-09-14 14:54:39 -07:00
  • 8031f6f8a3 Moar attempts = 2020-09-14 14:46:21 -07:00
  • d7a680a3ac Modify procfile instead of package.json for --inspect = 2020-09-14 07:17:37 -07:00
  • 8018caab47 Enable inspector and debugger = 2020-09-14 06:53:35 -07:00
  • d871d9ad1f More debugging, disable all crawlers Edin Dazdarevic 2020-09-13 23:45:08 +02:00
  • dfbefc20cd Remove PromisePool from olx = 2020-09-13 04:57:01 -07:00
  • a481ecfe37 Debug = 2020-09-13 04:48:11 -07:00
  • 8df94da48c Saljic fix fix Senad Uka 2020-09-11 05:03:37 +02:00
  • d4fcd1950d Fix saljic Senad Uka 2020-09-11 04:51:54 +02:00
  • e8115a9215 moar fixes = 2020-09-10 14:26:48 -07:00
  • 160efdf6ab More fixes = 2020-09-10 14:06:09 -07:00
  • c9b8c2e1a5 Fiks = 2020-09-10 13:31:57 -07:00
  • 855b93ca41 Scraped data Senad Uka 2020-09-10 19:46:25 +02:00
  • 70779b24c0 Scraped data Senad Uka 2020-09-10 19:42:50 +02:00
  • ba873f9f4e package json Senad Uka 2020-09-10 19:40:51 +02:00
  • e4775158fc Promise pool Senad Uka 2020-09-10 19:39:13 +02:00
  • 26377c485c Reverting Senad Uka 2020-09-10 18:30:14 +02:00
  • b30b0f45a6 trying different approach Senad Uka 2020-09-10 18:23:01 +02:00
  • 9c1a029ff1 Debugging ba Senad Uka 2020-09-10 18:17:16 +02:00
  • 9b49759485 Pause is not enough Senad Uka 2020-09-10 18:13:47 +02:00
  • 0c2f8d11ee Add random timeout up to 500ms Senad Uka 2020-09-10 18:08:24 +02:00
  • b27d9d3499 Add debugging info Senad Uka 2020-09-10 18:00:13 +02:00
  • dd3f30ef0e Comment agent header Senad Uka 2020-09-10 17:57:21 +02:00
  • 78c6056db4 Change for scraperapi Senad Uka 2020-09-10 17:34:39 +02:00
  • ecf27f2ba1 Merge branch 'improve-olx-scrapper' into 'master' Bilal Catic 2020-06-16 20:37:37 +00:00
  • 1229b3fa6c Add more debug logs Bilal 2020-06-16 19:51:13 +02:00
  • 542ff56123 Merge branch 'improve-wom' into 'master' Edin 2020-06-05 16:24:31 +00:00
  • 0aa851015b Improve styling of WOM message = 2020-06-05 09:07:54 -07:00
  • c033b2e47c Merge branch 'word-of-mouth-email' into 'master' Senad Uka 2020-06-05 14:55:44 +00:00
  • 0895654db2 Update emailContentGenerator.js Senad Uka 2020-06-05 14:55:00 +00:00
  • 8925eb9f4e Update emailContentGenerator.js Senad Uka 2020-06-05 14:54:22 +00:00
  • 52201af3ba Add word of mouth requests = 2020-06-05 07:52:52 -07:00
  • 1505c07363 Merge branch 'improve-crawler' into 'master' Senad Uka 2020-05-18 09:30:24 +00:00
  • 159fedbc2d handle failed page fetch Bilal 2020-05-18 03:53:08 +02:00
  • 65068932ad handle failed page fetch; detect discounted price Bilal 2020-05-18 03:43:49 +02:00
  • 820227827e Merge branch 'fix-crawler-errors' into 'master' Senad Uka 2020-05-14 21:34:17 +00:00
  • d35a113baa Fix saljic crawler Bilal 2020-05-14 19:01:19 +02:00
  • ba60f8749d Fix Prostor crawler - use new JSON location in page body Bilal 2020-05-14 15:38:15 +02:00
  • f1d45fed26 Merge branch 'move-scraper-url-to-the-env' into 'master' Senad Uka 2020-05-12 11:55:12 +00:00
  • ff923605ad Move scraper api base url to the ENV; send URL as base64 string Bilal 2020-05-12 13:44:09 +02:00
  • 11848cc0bb Added real estate ad edit ability email-notifications-kivi-ads Naida Vatric 2020-03-25 16:33:21 +01:00
  • 3fa9804ca6 Dropzone edit opction adadded. Naida Vatric 2020-03-24 11:10:16 +01:00
  • 477424caa1 Removing existing photos wip. Naida Vatric 2020-03-23 12:26:08 +01:00
  • 60f74c2cde Changed editing redirection. Naida Vatric 2020-03-20 15:40:38 +01:00
  • 55cb01c3c2 Started edit ad option. Naida Vatric 2020-03-19 15:24:58 +01:00
  • f2d9369d5c Added delete ability. Naida Vatric 2020-03-19 11:36:23 +01:00
  • ab50eb05af Started deleting of ads. Naida Vatric 2020-03-18 21:29:11 +01:00
  • 084766d0ea Added notification for new ad pusblish and new real estate. Naida Vatric 2020-03-16 16:13:16 +01:00
  • 981faeb610 For review view ad page. view-ad-kivi-original Naida Vatric 2020-03-14 15:34:18 +01:00
  • a36fba09eb Merged finished kivi-input in view Naida Vatric 2020-03-14 14:09:36 +01:00
  • 9f1fe3641d For revision kivi ads input. kivi-original-ads-input Naida Vatric 2020-03-12 22:46:47 +01:00
  • a7148ba6c3 Changed dropzone - url WiP. Naida Vatric 2020-03-11 22:32:10 +01:00
  • 5066c2fa70 Revert "Real estate input clean up." Naida Vatric 2020-03-10 22:03:32 +01:00
  • 5f674230e1 Real estate input clean up. Naida Vatric 2020-03-09 23:30:37 +01:00
  • 96bc66ef7b Dropzone implemented. Naida Vatric 2020-03-09 18:00:31 +01:00
  • 131536d9fb Olx added preflight check of available concurrent req. after-scraper-fix Naida Vatric 2020-03-06 14:25:02 +01:00
  • 824414adad Log for debugging of concurrent req. Naida Vatric 2020-03-05 18:05:37 +01:00
  • 41c926b5bb Debugging log olx - no normal price. Naida Vatric 2020-03-04 23:24:15 +01:00
  • b3708cf842 Fixed olx scraper api bug. Naida Vatric 2020-03-04 22:56:06 +01:00
  • 6821f61e55 Flexslider fixed. Naida Vatric 2020-03-03 22:03:41 +01:00
  • 54d9822fc8 Flexslider vs Materilize debugg. Naida Vatric 2020-03-03 17:02:20 +01:00
  • f5f8fa276c Created non-concurrent request groups. Naida Vatric 2020-03-03 15:21:51 +01:00
  • ccea5fe2aa Changed delay between ads. Naida Vatric 2020-03-01 00:44:48 +01:00
  • e1651306eb Changed log for olx. Naida Vatric 2020-02-29 19:09:50 +01:00
  • 97c09a6da1 Added delay between ads. Naida Vatric 2020-02-29 18:52:36 +01:00
  • 034106d87a Olx sraper scrape twice. Naida Vatric 2020-02-28 22:28:39 +01:00
  • 692577fb8c Merge branch 'after-scraper-fix' into 'master' Naida Vatric 2020-02-28 16:14:43 +00:00
  • df5e38092d Decomment and saljic smal fix. Naida Vatric 2020-02-28 17:12:55 +01:00
  • feb2d04ed6 Olx debugg log. Naida Vatric 2020-02-28 15:43:35 +01:00
  • 90e171d07d Olx scraper debugging. Naida Vatric 2020-02-28 15:41:20 +01:00
  • 747f56941a Logged olx price scraping. Naida Vatric 2020-02-28 14:21:53 +01:00
  • 2a13ab55ed Merge branch 'after-scraper-fix' into 'master' Naida Vatric 2020-02-27 22:38:00 +00:00
  • 441f905b29 Olx price parsing changed. Naida Vatric 2020-02-27 23:35:18 +01:00
  • 39f9383ae2 Merge branch 'after-scraper-fix' into 'master' Naida Vatric 2020-02-27 20:31:14 +00:00
  • edca7f91af Commented checkup out and Prostor fetch changed. Naida Vatric 2020-02-27 19:01:11 +01:00
  • 44402a9cc4 Merge branch 'saljic-crawler-fix' into 'master' Naida Vatric 2020-02-26 15:09:22 +00:00
  • bbd9dab30d Upload image file to bucket success. Naida Vatric 2020-02-26 14:56:00 +01:00
  • b913daa1f7 Merge branch 'checkup-email-bug-fix' into 'master' Naida Vatric 2020-02-25 15:13:20 +00:00
  • a508f72d7c Merge branch 'master' into 'checkup-email-bug-fix' checkup-email-bug-fix Naida Vatric 2020-02-25 15:12:47 +00:00
  • 08ad9edfe1 Merge branch 'scraper-api-support' into 'master' Naida Vatric 2020-02-25 15:10:39 +00:00
  • b80577ef6b WiP Ad preview page - flexslider problem. Naida Vatric 2020-02-24 23:15:48 +01:00
  • d36d7f413d Initialized flex slider Naida Vatric 2020-02-24 15:41:28 +01:00
  • ce857ddce9 Renamed var. scraper-api-support Naida Vatric 2020-02-23 23:11:21 +01:00
  • 9c63bdfbe2 Started photo gallery. Naida Vatric 2020-02-23 23:05:28 +01:00
  • 8435afe9c5 Renamed variables to describe purpose. rename-variables Naida Vatric 2020-02-23 18:47:27 +01:00
  • 148b2ea863 Changed default. Naida Vatric 2020-02-23 16:38:40 +01:00
  • 2218e6888a Added map to kivi ad preview. Naida Vatric 2020-02-23 01:52:43 +01:00
  • d436d4a37b Added Scraper API option. Naida Vatric 2020-02-22 22:15:27 +01:00
  • d7fcb2a278 Merge branch 'master' after user-agent change into email-density email-density Naida Vatric 2020-02-21 14:26:33 +01:00
  • 6bad24d735 New query for search req search. Naida Vatric 2020-02-21 14:25:10 +01:00
  • 6791a509d0 make user agent header configurable through env variable Bilal Catic 2020-02-20 21:07:16 +01:00
  • edc6e2bbf7 Merge branch 'create-fetch-wrapper-with-user-agent' into 'master' Bilal Catic 2020-02-20 19:58:32 +00:00
  • 4f230020d7 use fetch wrapper instead of node-fetch Bilal Catic 2020-02-20 19:49:29 +01:00