PhantomJS - Connect to GhostDriver Error

Question

I am scraping sites with PhantomJS and Selenium. My problem is that after about 50 verified URLs, I have an error:

selenium.common.exceptions.WebDriverException: Message: Cannot connect to GhostDriver

I don’t know how to fix it, I tried two versions of PhantomJS (1.9 and 1.98) and it does not work. Do you have any ideas?

Here is the code that I am executing:

def get_car_price(self, car_url): 
    browser = webdriver.PhantomJS('C:\phantomjs.exe') 
    browser.get(car_url) 
    content = browser.page_source 
    browser.quit() 

    website = lh.fromstring(content) 
    for price in website.xpath('//*[@id="js_item_' + str(self.car_id) + '"]/div[1]/div[2]/div[2]/strong[2]'): 
        return price.text

Answer 1

Instead of opening / exiting the browser, PhantomJSkeep it open and reuse. Create it all over the world when you run your script and close it when the script ends.

:

class Service(object):
    def __init__(self):
        self.browser = webdriver.PhantomJS('C:\phantomjs.exe') 

    def get_car_price(self, car_url): 
        self.browser.get(car_url) 
        content = self.browser.page_source 

        website = lh.fromstring(content) 
        for price in website.xpath('//*[@id="js_item_' + str(self.car_id) + '"]/div[1]/div[2]/div[2]/strong[2]'): 
            return price.text

    def shutdown(self):
        self.browser.quit() 

service = Service()
try:
    for url in urls:
        print(service.get_car_price(url))
finally:
    service.shutdown()