# kiviscrapworker

Kivi scraping worker

## Setup

Copy `example.env` to the `.env` and set desired values

## Deployment procedure

1. Compile source code with `go build`
2. Transfer generated binary to the clients

    Command for sending files to the remote computer : 
        
    `scp -i path/to/the/key.pem path/to/the/binary username@host:/path/to/destination`
    
    Path on the server where client resides is : `/home/ubuntu/kivi_scraper/client`    
    
3. Add `.env` file next to the binary on the server and set correct values
4. If `monit` is not installed on the machine where client worker will be installed, install it (`sudo apt install monit`)
5. Create `starth.sh` with contents 
        
        #!/bin/sh
        
        cd /home/ubuntu/kivi_scraper/client
        ./kiviscrapworker

6. Create `stop.sh` with contents
        
        #!/bin/sh
        
        cd /home/ubuntu/kivi_scraper/client
        sudo kill `cat sample.pid`
        
7. Edit `monit` configuration with `sudo nano /etc/monit/monitrc`
8. Around line 208 is example how to set process check. Add new line below example
9. Add to the file : 

       check process kivi_scraper_worker_client pidfile /home/ubuntu/kivi_scraper/client/sample.pid
         start program = "/home/ubuntu/kivi_scraper/client/start.sh"
         stop program  = "/home/ubuntu/kivi_scraper/client/stop.sh"
         if 5 restarts within 5 cycles then timeout

10. Reboot with `sudo reboot`

## Notes

* To check `monit` logs, check `/var/log/monit.log`