Puppeteer Page Load navigation waiting example
Puppeteer is a nodejs library that is used to test webpages in a headless browser. It opens chromium in a headless browser
Create a Directory puppeteertest
mkdir puppeteertest
cd puppeteertest
First, Create a nodejs project using the below command
npm init -y
this creates a nodejs project with the required basic configuration.
Install puppeteer dependency
npm install puppeteer --save
``
It installs the puppeteer into a node project
create a test.js file and add js code to it.
Let's see some use cases examples
## How to open a web page in Puppeteer
This is an example of opening a browser and closing the browser.
- Created a headless browser with `headless: false`, and launch the browser
- Create a new page using the browser.newPage();
- Go to url using the page
- Close the browsers
```markdown
const puppeteer = require('puppeteer');
async function test() {
const browser = await puppeteer.launch({
headless: false
});
const page = await browser.newPage();
const url = "https://developer.chrome.com/en/";
page
.waitForSelector('img')
.then(() => console.log('First URL with image: ' + url));
await page.goto(currentURL);
await browser.close();
}
test();
```
Running this file with the below command, with headless:base
option, we can see the browser running in a separate browser
node test.js
How to search and submit a form in puppeteer
This is a sample test of opening a browser, searching using keywords and loading the result, and closing the browser.
We have an amazon.in the website, Right click on the browser. First, find out below the selectors
For the search box class twotabsearchtextbox
Submit button has nav-search-submit-button
The following are steps to submit a form.
- Open a browser in a headless state using
headless: false
- Launch the browser using puppeteer.launch()
- Next, Create a new tab on the browser page.
- Open the amazon site on the web page using
page.goto()
- Find out selectors, In this case, We have used id selectors on the Amazon website.
- To emulate the typing in the text box, First, select the textbox using selectors. id selector used by appending # to selector and type to text
await page.type('#twotabsearchtextbox', 'mobile ');
- Next, Emulate search button click.
- Find out the search button selector, Id selector used
#nav-search-submit-button
- use page.click() with id selectors initiates a click-on button and results are loaded on the same page.
- wait for a page to load for 4 seconds using the
waitForTimeout
function - Finally, close the browser.
const puppeteer = require('puppeteer');
async function test() {
const browser = await puppeteer.launch({
headless: false
});
const page = await browser.newPage();
const url= "https://www.amazon.in/";
page
.waitForSelector('img')
.then(() => console.log('Found image: ' + url));
await page.goto(url);
await page.type('#twotabsearchtextbox', 'mobile ');
await page.click('#nav-search-submit-button');
await page.waitForTimeout(4000); // wait for 4 seconds
await browser.close();
}
test();
Puppeteer waits for page load after form submit
We have seen the page load process wait for specified timeout configured using page.waitForTimeout.
It is not a good idea to wait for a fixed timeout.
In this section, We will wait for the page to load.
How to know if the result page is loaded or not?
There are multiple ways we can know
One way using page.waitForNavigation(options)
Options are default empty and contain the below attributes
- timeout : assumes navigation completed for a given timeout
- waitUntil :
- load - Navigation completed when load event fired
- documentLoaded - this called similar to DOMContentLoaded fired
- networkidle0 : Navigation completed when there are zero network connections
- networkidle2 : Navigation completed when there 2 network connections.
We can use one the way based on your await page.waitForNavigation({waitUntil: ‘load’}); await page.waitForNavigation({waitUntil: ‘documentLoaded’}); await page.waitForNavigation({waitUntil: ‘networkidle0’}); await page.waitForNavigation({waitUntil: ‘networkidle2’});
Another way to use waitForSelector
, find out selector on the loaded page and use it with the below lines of code
await page.waitForSelector(".selector", { visible: true });
Here is a complete example
const puppeteer = require('puppeteer');
async function test() {
const browser = await puppeteer.launch({
headless: false
});
const page = await browser.newPage();
const url= "https://www.amazon.in/";
page
.waitForSelector('img')
.then(() => console.log('Found image: ' + url));
await page.goto(currentURL);
await page.type('#twotabsearchtextbox', 'mobile ');
await page.click('#nav-search-submit-button');
await page.waitForNavigation({ waitUntil: 'load' })
//await page.waitForNavigation({waitUntil: 'load'});
// await page.waitForNavigation({waitUntil: 'documentLoaded'});
//await page.waitForNavigation({waitUntil: 'networkidle0'});
//await page.waitForNavigation({waitUntil: 'networkidle2'});
//await page.waitForSelector(".selector", { visible: true });
console.log('page loaded')
await browser.close();
}
test();