Commit Graph

248 Commits

Author SHA1 Message Date
Bilal Catic
b3fcc6ba9a return new and existing real estates when saving results 2019-09-25 08:55:00 +02:00
Bilal Catic
f93d0e738f add delay between pages config variable 2019-09-25 08:55:00 +02:00
Bilal Catic
90bc57edb6 stop crawling when existing, non-renewed ad is found 2019-09-25 08:55:00 +02:00
Bilal Catic
746732f30b remove deleted column from RealEstate model 2019-09-25 08:55:00 +02:00
Bilal Catic
06d35fcb4b move ignored usernames config to crawler specific config 2019-09-25 08:55:00 +02:00
Bilal Catic
63eb64b0f6 parse and save published and renewed dates 2019-09-25 08:55:00 +02:00
Bilal Catic
c7184be5fc install moment and moment-timezone packages 2019-09-25 08:55:00 +02:00
Bilal Catic
18db554ea8 add published and renewed date columns to the RealEstates table 2019-09-25 08:55:00 +02:00
Bilal Catic
3140fdf0c0 use function generator to index pages; crawl in parallel 2019-09-25 08:55:00 +02:00
Bilal Catic
c4f6c6e1c3 construct crawling url before indexing single page 2019-09-25 08:55:00 +02:00
Edin Dazdarevic
5f1697f6ae Set viewport correctly 2019-09-21 19:44:22 +02:00
Bilal Catic
7845587fa7 Merge branch 'go_again_copy' into 'master'
Change final screen to explain to user what they can expect next

See merge request saburly/marketalarm/web!32
2019-09-21 15:58:20 +00:00
Bilal Catic
c2d1725173 Merge branch 'map_screen_copy' into 'master'
Add copy that explains the map screen better.

See merge request saburly/marketalarm/web!31
2019-09-21 14:42:32 +00:00
Bilal Catic
e3636451b7 fix typo 2019-09-21 14:41:20 +00:00
Senad Uka
0f01a070ca Change final screen to explain to user what they can expect next 2019-09-21 08:14:04 +02:00
Senad Uka
bfcc0a09d9 Add copy that explains the map screen better. 2019-09-21 06:06:39 +02:00
Bilal Catic
51411a4109 Merge branch 'switch-to-new-crawler' into 'master'
Switch to new crawler

See merge request saburly/marketalarm/web!30
2019-09-18 14:06:16 +00:00
Bilal Catic
f9cd26fca8 refactor enums 2019-09-18 15:33:58 +02:00
Bilal Catic
e374cac60e update RealEstates model, add DB helper functions for RealEstates model 2019-09-18 15:33:42 +02:00
Bilal Catic
d3b2f95719 add new ENV variables to control crawler 2019-09-18 15:33:09 +02:00
Bilal Catic
3d46c82d3d create new crawler and Postgres saver 2019-09-18 15:32:48 +02:00
Bilal Catic
2e13763939 update Sequelize 2019-09-18 06:35:39 +02:00
Bilal Catic
2702259eb1 change title, short and long description columns type to Text 2019-09-18 06:35:26 +02:00
Bilal Catic
17461b2e57 remove deleted and sold columns, adStatus is replacement column 2019-09-17 10:27:25 +02:00
Bilal Catic
5cb12b1714 remove lastTimeCrawled column - use updatedAt for this 2019-09-17 10:12:49 +02:00
Bilal Catic
b96ce17dbf add unique constraint - composite key to the RealEstates table 2019-09-17 09:07:14 +02:00
Bilal Catic
88ee2ee326 add adStatus column to the RealEstates table 2019-09-16 23:20:18 +02:00
Bilal Catic
33b91c8161 add short and long description columns to the RealEstates table 2019-09-16 23:09:29 +02:00
Bilal Catic
25faf75d6f add title column to the RealEstates table 2019-09-16 16:08:29 +02:00
Bilal Catic
76a989fa37 replace old crawler, without specific crawler and saver implementation 2019-09-16 15:59:53 +02:00
Bilal Catic
b28b160852 fix locate me button position 2019-09-16 11:56:55 +02:00
Bilal Catic
dbb9807e63 Merge branch 'redesign-DB-and-adapt-search-request' into 'master'
Redesign db and adapt search request

See merge request saburly/marketalarm/web!29
2019-09-13 16:55:11 +00:00
Bilal Catic
5dfe363adc adapt unsubscribe step for new DB design 2019-09-13 14:54:21 +02:00
Bilal Catic
2b22cd04fd fix unsubscribe and review links in email notification 2019-09-13 14:50:23 +02:00
Bilal Catic
c01cb03762 adapt query submit size step for new DB design 2019-09-13 14:40:10 +02:00
Bilal Catic
b4c71b93de adapt garden size step for new DB design 2019-09-13 14:30:54 +02:00
Bilal Catic
e26c2b6e8d adapt price and query review steps for new DB design 2019-09-13 14:17:46 +02:00
Bilal Catic
ff68e96f4f adapt size step for new DB design 2019-09-13 13:58:58 +02:00
Bilal Catic
db58d1e98b adapt location step for new DB design 2019-09-13 12:37:53 +02:00
Bilal Catic
fbf7aabe93 use sequelize findByPk method for getting SearchRequest object 2019-09-13 12:35:23 +02:00
Bilal Catic
f258800fd8 change defaultValue for subscribed to false 2019-09-13 12:33:16 +02:00
Bilal Catic
8f09d4f227 change url param case from snake_case to camelCase 2019-09-13 11:11:28 +02:00
Bilal Catic
1a874d4d88 adapt first step of search request to new DB design 2019-09-13 11:08:45 +02:00
Bilal Catic
3cbd79dcc9 create SearchRequest db helper model 2019-09-13 11:06:58 +02:00
Bilal Catic
77d9669ad6 modify URL helper for getting searchRequestId 2019-09-13 11:06:03 +02:00
Bilal Catic
b7d147b0a6 fix SearchRequest model, add default values for not null fields 2019-09-13 10:57:43 +02:00
Bilal Catic
e32e98537c add createdAt, updatedAt fields; add default values for not null fields 2019-09-13 10:56:46 +02:00
Bilal Catic
81ecb37493 add models for new DB design 2019-09-13 09:47:49 +02:00
Bilal Catic
75daf55fdf add migrations for new DB design 2019-09-13 09:46:59 +02:00
Bilal Catic
05f8cbd816 move old models to oldModels folder 2019-09-13 08:57:41 +02:00