Puppeteer Page Load navigation waiting example

Puppeteer Page Load navigation waiting example

Puppeteer is a nodejs library that is used to test webpages in a headless browser. It opens chromium in a headless browser

Create a Directory puppeteertest

mkdir puppeteertest
cd puppeteertest

First, Create a nodejs project using the below command

npm init -y

this creates a nodejs project with the required basic configuration.

Install puppeteer dependency

npm install puppeteer --save
``
It installs the puppeteer into a node project

create a test.js file and add js code to it.

Let's see some use cases examples

## How to open a web page in Puppeteer

This is an example of opening a browser and closing the browser.

- Created a headless browser with `headless: false`, and launch the browser
- Create a new page using the browser.newPage();
- Go to url using the page
- Close the browsers

```markdown
const puppeteer = require('puppeteer');
async function test() {
const browser = await puppeteer.launch({
headless: false
});
const page = await browser.newPage();
const url = "https://developer.chrome.com/en/";
page
.waitForSelector('img')
.then(() => console.log('First URL with image: ' + url));

await page.goto(currentURL);
await browser.close();

}
test();
```

Running this file with the below command, with headless:base option, we can see the browser running in a separate browser

node test.js

How to search and submit a form in puppeteer

This is a sample test of opening a browser, searching using keywords and loading the result, and closing the browser.

We have an amazon.in the website, Right click on the browser. First, find out below the selectors For the search box class twotabsearchtextbox Submit button has nav-search-submit-button

The following are steps to submit a form.

  • Open a browser in a headless state using headless: false
  • Launch the browser using puppeteer.launch()
  • Next, Create a new tab on the browser page.
  • Open the amazon site on the web page using page.goto()
  • Find out selectors, In this case, We have used id selectors on the Amazon website.
  • To emulate the typing in the text box, First, select the textbox using selectors. id selector used by appending # to selector and type to text
await page.type('#twotabsearchtextbox', 'mobile ');
  • Next, Emulate search button click.
  • Find out the search button selector, Id selector used #nav-search-submit-button
  • use page.click() with id selectors initiates a click-on button and results are loaded on the same page.
  • wait for a page to load for 4 seconds using the waitForTimeout function
  • Finally, close the browser.
const puppeteer = require('puppeteer');
async function test() {
const browser = await puppeteer.launch({
headless: false
});
const page = await browser.newPage();
const url= "https://www.amazon.in/";
page
.waitForSelector('img')
.then(() => console.log('Found image: ' + url));
await page.goto(url);
await page.type('#twotabsearchtextbox', 'mobile ');
await page.click('#nav-search-submit-button');
await page.waitForTimeout(4000); // wait for 4 seconds

await browser.close();

}
test();

Puppeteer waits for page load after form submit

We have seen the page load process wait for specified timeout configured using page.waitForTimeout.

It is not a good idea to wait for a fixed timeout.

In this section, We will wait for the page to load.

How to know if the result page is loaded or not?

There are multiple ways we can know

One way using page.waitForNavigation(options) Options are default empty and contain the below attributes

  • timeout : assumes navigation completed for a given timeout
  • waitUntil :
    • load - Navigation completed when load event fired
    • documentLoaded - this called similar to DOMContentLoaded fired
    • networkidle0 : Navigation completed when there are zero network connections
    • networkidle2 : Navigation completed when there 2 network connections.

We can use one the way based on your await page.waitForNavigation({waitUntil: ‘load’}); await page.waitForNavigation({waitUntil: ‘documentLoaded’}); await page.waitForNavigation({waitUntil: ‘networkidle0’}); await page.waitForNavigation({waitUntil: ‘networkidle2’});

Another way to use waitForSelector, find out selector on the loaded page and use it with the below lines of code

await page.waitForSelector(".selector", { visible: true });

Here is a complete example

const puppeteer = require('puppeteer');
async function test() {
const browser = await puppeteer.launch({
headless: false
});
const page = await browser.newPage();
const url= "https://www.amazon.in/";
page
.waitForSelector('img')
.then(() => console.log('Found image: ' + url));

    await page.goto(currentURL);

await page.type('#twotabsearchtextbox', 'mobile ');
await page.click('#nav-search-submit-button');
await page.waitForNavigation({ waitUntil: 'load' })

//await page.waitForNavigation({waitUntil: 'load'});
// await page.waitForNavigation({waitUntil: 'documentLoaded'});
//await page.waitForNavigation({waitUntil: 'networkidle0'});
//await page.waitForNavigation({waitUntil: 'networkidle2'});
//await page.waitForSelector(".selector", { visible: true });

console.log('page loaded')

await browser.close();

}
test();