49 lines
1.5 KiB
Markdown
49 lines
1.5 KiB
Markdown
# kiviscrapworker
|
|
|
|
Kivi scraping worker
|
|
|
|
## Setup
|
|
|
|
Copy `example.env` to the `.env` and set desired values
|
|
|
|
## Deployment procedure
|
|
|
|
1. Compile source code with `go build`
|
|
2. Transfer generated binary to the clients
|
|
|
|
Command for sending files to the remote computer :
|
|
|
|
`scp -i path/to/the/key.pem path/to/the/binary username@host:/path/to/destination`
|
|
|
|
Path on the server where client resides is : `/home/ubuntu/kivi_scraper/client`
|
|
|
|
3. Add `.env` file next to the binary on the server and set correct values
|
|
4. If `monit` is not installed on the machine where client worker will be installed, install it (`sudo apt install monit`)
|
|
5. Create `starth.sh` with contents
|
|
|
|
#!/bin/sh
|
|
|
|
cd /home/ubuntu/kivi_scraper/client
|
|
./kiviscrapworker
|
|
|
|
6. Create `stop.sh` with contents
|
|
|
|
#!/bin/sh
|
|
|
|
cd /home/ubuntu/kivi_scraper/client
|
|
sudo kill `cat sample.pid`
|
|
|
|
7. Edit `monit` configuration with `sudo nano /etc/monit/monitrc`
|
|
8. Around line 208 is example how to set process check. Add new line below example
|
|
9. Add to the file :
|
|
|
|
check process kivi_scraper_worker_client pidfile /home/ubuntu/kivi_scraper/client/sample.pid
|
|
start program = "/home/ubuntu/kivi_scraper/client/start.sh"
|
|
stop program = "/home/ubuntu/kivi_scraper/client/stop.sh"
|
|
if 5 restarts within 5 cycles then timeout
|
|
|
|
10. Reboot with `sudo reboot`
|
|
|
|
## Notes
|
|
|
|
* To check `monit` logs, check `/var/log/monit.log` |