# kiviscrapbalancer

Kivi scraping load balancer

## Setup

Copy `example.env` to the `.env` and set desired values

## Deployment procedure

1. Compile source code with `go build`
2. Transfer generated binary to the Kivi load balancer server
    
    Command for sending files to the remote computer : 
    
    `scp -i path/to/the/key.pem path/to/the/binary username@host:/path/to/destination`
    
    Path on the server where load balancer resides is : `/home/ubuntu/kivi_scraper/load_balancer`
    
3. Add `.env` file next to the binary on the server and set correct values
    
    Note: for listening addresses, use `:1337` and `:1338` or other ports but in this format, so the server can be reached from outside
    
    Note : make sure firewalls(on the server and one from the platform where server is hosted) is not blocking selected ports
4. If `monit` is not installed on the server, install it (`sudo apt install monit`)
5. Create `starth.sh` with contents 
        
        #!/bin/sh
        
        cd /home/ubuntu/kivi_scraper/load_balancer
        ./kiviscrapbalancer

6. Create `stop.sh` with contents
        
        #!/bin/sh
        
        cd /home/ubuntu/kivi_scraper/load_balancer
        sudo kill `cat sample.pid`
        
7. Edit `monit` configuration with `sudo nano /etc/monit/monitrc`
8. Around line 208 is example how to set process check. Add new line below example
9. Add to the file : 

           check process kivi_load_balancer with pidfile /home/ubuntu/kivi_scraper/load_balancer/sample.pid
             start program = "/home/ubuntu/kivi_scraper/load_balancer/start.sh"
             stop program  = "/home/ubuntu/kivi_scraper/load_balancer/stop.sh"
             if failed port 1337 protocol http then restart
             if failed port 1338 then restart
             if 5 restarts within 5 cycles then timeout

10. Reboot with `sudo reboot`

## Notes

* To check `monit` logs, check `/var/log/monit.log`

* To check on which ports system is listening, use `sudo lsof -i -P -n | grep LISTEN`