"Facebook \ WebDriver \ Exception \ TimeoutException" when using waitFor() in symfony/panther & Laravel

Question

I have a laravel app that is using symfony/panther to scrape js loaded web pages.

Upon running, I always get a timeout exception Facebook \ WebDriver \ Exception \ TimeoutException at the waitFor() method.

I've tried increasing the max_execution_time in my php.ini file and also tried doing this in a queue, but I still get the same error

To test I'm using a livewire component

public function handle()

{

    $client = Client::createChromeClient(base_path("drivers/chromedriver"), null, ["port" => 9080]);    // create a chrome client

    $crawler = $client->request('GET', 'https://example.com');

    $client->waitFor('h1'); // Fails here ***********
    
    $crawler->filter('h1')->text();


}

Any ideas what could solve this timeout exception? Thanks!

score 1 · Accepted Answer · answered Jun 28 '22 at 21:31

Turns out the website being visited was not loading the JS due to console errors. Therefore the waitFor() never sees its element. The errors were CORS Policy errors and 403 errors. Both due to the browser being automated. Incase you have stumbled on this, the solution that worked for me is to add these flags:

$client = Client::createChromeClient(base_path("drivers/chromedriver"), [
    '--user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36',
    '--window-size=1200,1100',
    '--disable-web-security',
    '--disable-blink-features=AutomationControlled',
], ["port" => 9080,]);

"Facebook \ WebDriver \ Exception \ TimeoutException" when using waitFor() in symfony/panther & Laravel

1 Answers1