I have a problem with Aquarium and splash. They stop working 30 minutes after the start. The number of pages to download is 50K-80K. I did a cron job to automatically reload every 10 minutes, every Splash container, but it didn’t work. How can I fix this?
Here is a screenshot
and statistics from HAProxy
Here is the Splash configuration
splash0: image: scrapinghub/splash:3.0 command: --max-timeout 3600 --slots 150 --maxrss 1000 --verbosity 5 logging: driver: "none" expose: - 8050 mem_limit: 1000m memswap_limit: 1000m restart: always
And HAProxy
backend splash-cluster option httpchk GET / balance leastconn
UPDATE 1 Here's a script to reload
#!/bin/sh echo "BEGIN" >> restart.log for index in 0 1 2 3 4 5 6 7 do docker-compose restart splash$index echo "Restarted: Splash$index. Time: $(date '+%A %W %Y %X')" >> restart.log sleep 1 done echo "END" >> restart.log
And it should work because I have the logs in the Cronjob file:
cd /var/www/project/aquarium && sh splash_restart.sh
source share