Commit Graph

271 Commits

Author SHA1 Message Date
Bilal Catic
dac7043a56 add service to find real estate - search request matches 2019-09-30 12:07:47 +02:00
Bilal Catic
12f826ef55 add methods to add and find real estate - search request matches 2019-09-30 12:07:21 +02:00
Bilal Catic
5a7b9826c2 refactor email content 2019-09-30 12:06:35 +02:00
Bilal Catic
b3ea2398d4 add more real estates to test 2019-09-30 10:31:50 +02:00
Bilal Catic
af36653c1a refactor redirect - use new method to find real estate by id 2019-09-30 10:31:07 +02:00
Bilal Catic
31ffc3253f add method to find search requests for real estate 2019-09-30 10:30:13 +02:00
Bilal Catic
54f7e4ba53 add method to find real estate by id 2019-09-30 10:29:27 +02:00
Bilal Catic
87a72be8fe use notificationService to send email for new search request 2019-09-30 10:29:05 +02:00
Bilal Catic
a738a371ae add notificationService to handle when, what and to whom to send email 2019-09-30 10:28:29 +02:00
Bilal Catic
9c0104a57c refactor crawler - adapt to use new ENUM objects 2019-09-30 10:27:12 +02:00
Bilal Catic
e1dfd6a4eb add helper to generate email; install html-to-text package 2019-09-30 09:51:10 +02:00
Bilal Catic
68ef31f867 add emailService to handle email sending 2019-09-30 09:49:47 +02:00
Bilal Catic
e3e47345bc load AWS config through app config; fix ENV path 2019-09-30 09:44:19 +02:00
Bilal Catic
c57d3f8457 add npm command to test matching and notification services 2019-09-29 19:28:47 +02:00
Bilal Catic
6d7366f2c2 use single ENUMS; change old realEstateType enums 2019-09-27 19:36:20 +02:00
Bilal Catic
d8d801d314 use single ENUMS; change old realEstateType enums 2019-09-27 18:20:16 +02:00
Bilal Catic
f31355dd84 use AD_TYPE enums for SearchRequest adType field 2019-09-27 16:49:10 +02:00
Bilal Catic
ad873d61e7 Merge branch 'start-crawler-loop-with-server' into 'master'
start crawler loop when server is started

See merge request saburly/marketalarm/web!34
2019-09-27 09:22:17 +00:00
Bilal Catic
c5a720484a add ENV variable to control crawler execution 2019-09-26 23:55:34 +02:00
Bilal Catic
2e92f961ff start crawler loop when server is started 2019-09-26 17:30:06 +02:00
Bilal Catic
0b083a02e2 Merge branch 'make-crawler-smarter' into 'master'
Make crawler smarter

See merge request saburly/marketalarm/web!33
2019-09-25 17:15:14 +00:00
Bilal Catic
3d203df988 remove comment from delay between indexing pages 2019-09-25 10:00:42 +00:00
Bilal Catic
c9a959f8be stop crawling when existing, not renewed ad is found 2019-09-25 08:55:00 +02:00
Bilal Catic
b3fcc6ba9a return new and existing real estates when saving results 2019-09-25 08:55:00 +02:00
Bilal Catic
f93d0e738f add delay between pages config variable 2019-09-25 08:55:00 +02:00
Bilal Catic
90bc57edb6 stop crawling when existing, non-renewed ad is found 2019-09-25 08:55:00 +02:00
Bilal Catic
746732f30b remove deleted column from RealEstate model 2019-09-25 08:55:00 +02:00
Bilal Catic
06d35fcb4b move ignored usernames config to crawler specific config 2019-09-25 08:55:00 +02:00
Bilal Catic
63eb64b0f6 parse and save published and renewed dates 2019-09-25 08:55:00 +02:00
Bilal Catic
c7184be5fc install moment and moment-timezone packages 2019-09-25 08:55:00 +02:00
Bilal Catic
18db554ea8 add published and renewed date columns to the RealEstates table 2019-09-25 08:55:00 +02:00
Bilal Catic
3140fdf0c0 use function generator to index pages; crawl in parallel 2019-09-25 08:55:00 +02:00
Bilal Catic
c4f6c6e1c3 construct crawling url before indexing single page 2019-09-25 08:55:00 +02:00
Edin Dazdarevic
5f1697f6ae Set viewport correctly 2019-09-21 19:44:22 +02:00
Bilal Catic
7845587fa7 Merge branch 'go_again_copy' into 'master'
Change final screen to explain to user what they can expect next

See merge request saburly/marketalarm/web!32
2019-09-21 15:58:20 +00:00
Bilal Catic
c2d1725173 Merge branch 'map_screen_copy' into 'master'
Add copy that explains the map screen better.

See merge request saburly/marketalarm/web!31
2019-09-21 14:42:32 +00:00
Bilal Catic
e3636451b7 fix typo 2019-09-21 14:41:20 +00:00
Senad Uka
0f01a070ca Change final screen to explain to user what they can expect next 2019-09-21 08:14:04 +02:00
Senad Uka
bfcc0a09d9 Add copy that explains the map screen better. 2019-09-21 06:06:39 +02:00
Bilal Catic
51411a4109 Merge branch 'switch-to-new-crawler' into 'master'
Switch to new crawler

See merge request saburly/marketalarm/web!30
2019-09-18 14:06:16 +00:00
Bilal Catic
f9cd26fca8 refactor enums 2019-09-18 15:33:58 +02:00
Bilal Catic
e374cac60e update RealEstates model, add DB helper functions for RealEstates model 2019-09-18 15:33:42 +02:00
Bilal Catic
d3b2f95719 add new ENV variables to control crawler 2019-09-18 15:33:09 +02:00
Bilal Catic
3d46c82d3d create new crawler and Postgres saver 2019-09-18 15:32:48 +02:00
Bilal Catic
2e13763939 update Sequelize 2019-09-18 06:35:39 +02:00
Bilal Catic
2702259eb1 change title, short and long description columns type to Text 2019-09-18 06:35:26 +02:00
Bilal Catic
17461b2e57 remove deleted and sold columns, adStatus is replacement column 2019-09-17 10:27:25 +02:00
Bilal Catic
5cb12b1714 remove lastTimeCrawled column - use updatedAt for this 2019-09-17 10:12:49 +02:00
Bilal Catic
b96ce17dbf add unique constraint - composite key to the RealEstates table 2019-09-17 09:07:14 +02:00
Bilal Catic
88ee2ee326 add adStatus column to the RealEstates table 2019-09-16 23:20:18 +02:00