Tips for End to End Testing with Puppeteer

Getting up to speed with Puppeteer!

08.06.2020 — JavaScript, Software Testing — 5 min read

This article is available in Korean | 한국어 가능

These are exciting times for End to End (E2E) testing in the JavaScript world. In the last couple of years, tools such as cypress and Puppeteer have flooded the JavaScript community and gain a fast adoption.

Today I'm writing about Puppeteer.

I want to share a pragmatic list of tips and resources that can help you get a fast overall understanding of things to consider when using Puppeteer, and what it has to offer.

Getting things running

In this section, I discuss the main aspects of running a test with Puppeteer, including some interoperability aspects that we should consider, such as the usage of an underlying testing library/framework such as Jest.

Running tests in parallel

To launch different browser instances to run your test suite, you can rely on your chosen test runner. For example, with Jest, I leverage the config maxWorkers to define how many browser sessions I allow to run concurrently.

Be aware of the global timeout value

You want to increase the default global value for a test to timeout. E2E tests might take up several seconds to run. If you're using Jest, you can configure the timeout value with the property testTimeout, which for Jest 26.0 defaults to 5 seconds.

Here's an example of my jest.config.js with the mentioned configurations.

jest.config.js

1module.exports = {
2  verbose: true,
3  rootDir: ".",
4  testTimeout: 30000,
5  maxWorkers: 3,
6};

If you're using (for example) mocha, you can add this.timeout(VALUE_IN_SECONDS); at the top level of your describe block.

Abstracting puppeteer.launch

To bootstrap your test, you have to run puppeteer.lauch. I recommend you to be abstract this call within a wrapper function. Doing so allows you to centralize all your test environment customizations easily. I'm referring to making the following things configurable:

Allow the client to specify what page the browser should open on.
Allow the client to decide under what network conditions the test runs. I'll also cover this in this article.
Allow the client to specify things like whether the DevTools are open etc.

boot.js

1import puppeteer from "puppeteer";
2
3export default async function boot(options = {}) {
4  let page = null;
5  let browser = null;
6
7  const { goToTargetApp = true, headless = true, devtools = false, slowMo = false } = options;
8
9  browser = await puppeteer.launch({
10    headless,
11    devtools,
12    ...(slowMo && { slowMo }),
13  });
14
15  if (goToTargetApp) {
16    page = await browser.newPage();
17    // I'm assuming there's some environment variable here
18    // that points towards the app we're going to test
19    await page.goto(process.env.APP_URL);
20  }
21
22  return { page };
23}

I like to have my launch function just dealing with the bootstrap configuration aspects of my test environment and launch the application. I try to keep it as slimmer as possible, but sometimes I feel the urge to add more stuff here. There's a saying:

"Functions should do one thing. They should do it well. They should do it only."

source: Clean Code by Robert C. Martin

Throttling Network Connection Speed

You can run your tests under different network speed conditions. Let me share the pattern I use based on this gist that I luckily found.

If you abstract puppeteer.launch, your test could switch between network presets just by doing the following.

boot.js

1import puppeteer from "puppeteer";
2import NETWORK_PRESETS from "./network-presets";
3
4export default async function boot(options = {}) {
5  let page = null;
6  let browser = null;
7
8  const { goToTargetApp = true, headless = true, devtools = false, slowMo = false } = options;
9
10  browser = await puppeteer.launch({
11    headless,
12    devtools,
13    ...(slowMo && { slowMo }),
14  });
15
16  if (goToTargetApp) {
17    page = await browser.newPage();
18    // I'm assuming there's some environment variable here
19    // that points towards the app we're going to test
20    await page.goto(`${process.env.TARGET_APP_URL}${targetAppQueryParams}`);
21
22    if (network && NETWORK_PRESETS[network]) {
23      // setup custom network speed
24      const client = await page.target().createCDPSession();
25      await client.send("Network.emulateNetworkConditions", NETWORK_PRESETS[network]);
26    }
27  }
28
29  return { page };
30}

network-presets.js

1// source: https://gist.github.com/trungpv1601/2ccd3cc998149a84ba80ed7a4c9ef562
2export default {
3  GPRS: {
4    offline: false,
5    downloadThroughput: (50 * 1024) / 8,
6    uploadThroughput: (20 * 1024) / 8,
7    latency: 500,
8  },
9  Regular2G: {
10    offline: false,
11    downloadThroughput: (250 * 1024) / 8,
12    uploadThroughput: (50 * 1024) / 8,
13    latency: 300,
14  },
15  Good2G: {
16    offline: false,
17    downloadThroughput: (450 * 1024) / 8,
18    uploadThroughput: (150 * 1024) / 8,
19    latency: 150,
20  },
21  Regular3G: {
22    offline: false,
23    downloadThroughput: (750 * 1024) / 8,
24    uploadThroughput: (250 * 1024) / 8,
25    latency: 100,
26  },
27  Good3G: {
28    offline: false,
29    downloadThroughput: (1.5 * 1024 * 1024) / 8,
30    uploadThroughput: (750 * 1024) / 8,
31    latency: 40,
32  },
33  Regular4G: {
34    offline: false,
35    downloadThroughput: (4 * 1024 * 1024) / 8,
36    uploadThroughput: (3 * 1024 * 1024) / 8,
37    latency: 20,
38  },
39  DSL: {
40    offline: false,
41    downloadThroughput: (2 * 1024 * 1024) / 8,
42    uploadThroughput: (1 * 1024 * 1024) / 8,
43    latency: 5,
44  },
45  WiFi: {
46    offline: false,
47    downloadThroughput: (30 * 1024 * 1024) / 8,
48    uploadThroughput: (15 * 1024 * 1024) / 8,
49    latency: 2,
50  },
51};

Loading a Browser Extension

Here's how you can load a browser extension.

1// 1. launch puppeeter pass along the EXTENSION_PATH within your project
2// a relative path that points to the directory you output your extension assets
3browser = await puppeteer.launch({
4  // extension are allowed only in head-full mode
5  headless: false,
6  devtools,
7  args: [`--disable-extensions-except=${process.env.EXTENSION_PATH}`, `--load-extension=${process.env.EXTENSION_PATH}`],
8  ...(slowMo && { slowMo }),
9});
10
11// 2. find the extension by the title
12// you might want to tackle this differently
13// depending on your use case
14const targets = await browser.targets();
15const extensionTarget = targets.find(({ _targetInfo }) => {
16  return _targetInfo.title === "my extension page title";
17});
18
19// 3. getting the extensionId from the URL
20// if you have a fixed extensionId you can just pass in an
21// environment variable with that value, otherwise this works fine
22const partialExtensionUrl = extensionTarget._targetInfo.url || "";
23const [, , extensionID] = partialExtensionUrl.split("/");
24// here the entry point of the extension is an html file called "popup.html"
25const extensionPopupHtml = "popup.html";
26
27// 4. open the chrome extension in a new tab
28// notice that to properly build the extension URL you need the
29// extensionId and the entrypoint resource
30extensionPage = await browser.newPage();
31extensionUrl = `chrome-extension://${extensionID}/${extensionPopupHtml}`;
32
33await extensionPage.goto(extensionUrl);
34
35// ... now use extensionPage to interact with the extension

If you want to read through about testing chrome extensions with Puppeteer, I recommend this article: Automate the UI Testing of your chrome extension by Gokul Kathirvel.

Writing Tests

Apart from the last subsection, what I discuss next, can be easily found in the official documentation. I'm just going to step on those topics that I consider to be essential parts of the Puppeteer API.

Working with page.evaluate

You'll need to get used to the detail that when using page.evaluate, you run on the page context, meaning even if you're using arrow functions as an argument to page.evaluate, you can't refer to things out of the scope of that function. You need to provide all the data you'll need as the third argument of page.evaluate. Keep this in mind.

1// extracting the "value" from an input element
2const inputValue = await inputEl.evaluate((e) => e.value);

page.waitForSelector & page.waitForFunction

Quickly getting familiar with the APIs page.waitForSelector and page.waitForFunction can reveal itself very productive. If you have a couple of tests to write changes that you'll need to wait for some condition to be met in the UI before you allowing your test to proceed, are high. Suspend the test flow and wait for the UI is a common practice, not exclusive to Puppeteer. See the below examples for some basic usages.

1// this function waits for the menu to appear before
2// proceeding, this way we can ensure that  we can interact
3// with the list items in the menu
4const getSmuiSelectOptions = async () => {
5  const selector = ".mdc-menu-surface li";
6  await page.waitForSelector(selector, { timeout: 1000 });
7  return await page.$$(selector);
8};

1// wait for a snackbar to appear when some item is deleted in the application
2await extensionPage.waitForFunction(
3  () => !!document.querySelector('*[data-testid="global-snackbar"]').innerText.includes("deleted"),
4  {
5    timeout: 2000,
6  },
7);

There's a decision you need to make. The choice is whereas you should have a higher or lower timeout. I usually try to advocate for lower as possible, because we want to keep our tests fast. Running E2E tests against systems where you need to perform (not mocked) network requests, means that you need to account for network instability, altough usually you'll run under perfect network conditions, you might want to cut some slack to the timeout value.

element.select

I like the way it's possible to select options on a native HTML select element. It works both for single and multiple selections, and it feels natural.

1// selecting an HTTP method in a select element with id "custom-http-method"
2const selectEl = await page.$("#custom-http-method");
3
4await selectEl.select("POST");

While element.select it's convenient, you'll probably have to approach this differently for custom select fields built on a div > ul > li structure with a hidden input field, for instance select Material UI components.

Screenshots

For specific test cases, I like to output a collection of screenshots that build a timeline of how my application looks throughout the test. Screenshotting in between your test helps you get an initial pointer to what you should be debugging in a failing test. Here's my small utility that wraps page.screenshot API.

1// wrapping the call to `page.screenshot` just to avoid it
2// breaking my test in case the screenshot fails
3export async function prtScn(page, path = `Screenshot ${new Date().toString()}`) {
4  try {
5    await page.screenshot({ path, type: "png", fullPage: true });
6  } catch (error) {
7    // eslint-disable-next-line no-console
8    console.error(error);
9    // eslint-disable-next-line no-console
10    console.info("Failed to take screenshot but test will proceed...");
11    return Promise.resolve();
12  }
13}

1import * as utils from "test-utils";
2
3// (...)
4
5// Note: you can make this utility a Class and pass along the page
6// as context so that you don't need to pass it in everytime you
7// need to take a screenshot
8await utils.prtScn(page);

Page Reloads

With the page.reload API. You can specify a set of options that allow you to wait for specific underlying browser tasks to idle before proceeding.

1await page.reload({ waitUntil: ["networkidle0"] });

In this above example, we reload the page with networkidle0, which does not allow the test to proceed unless there are no HTTP requests within a half a second period.

Clearing Text from an Input Field

I was stunned not to find a very out-of-the-box way to clear an input field. A few developers have expressed interest in this feature, but it seems there's no interest on the other end. I've found a way to do it:

1/**
2 * Clears an element
3 * @param {ElementHandle} el
4 */
5export async function clear(el) {
6  await el.click({ clickCount: 3 });
7  await el.press("Backspace");
8}

It only works on Chrome as it takes advantage of the functionality where three consecutive clicks in a text area/input field select the whole text. After that, you need to trigger a keyboard event to clear the entire field.

Debugging

I want to highlight some debugging techniques. Especially the slowMo option.

Debugging with slowMo

You'll want to use slowMo to debug individual tests. The option allows you to slow down the interactions (the steps) of your E2E test so that you can see what's going on, almost like seeing an actual human interacting with your application. I can't emphasize enough how valuable this is.

1page.launch({ slowMo: 50 });

In the following GIFs you can see the difference of running without and with the slowMo option respectivelly.

E2E test on the tweak chrome extension without slowMo. You can't possibly understand what's going on.

E2E test on the tweak chrome extension with slowMo. You can see the characters appearing in the 'URL' as if a human would be typing.

In these examples I'm using the tweak browser extension to demo the different use cases.

For more awesome tips for debugging, I highly recommend this short article on Debugging Tips from Google.

Using debugger

I got this one from Google debugging tips. I had the habit of throwing a sleep statement to stop my tests for X seconds and inspect the application to see why the tests were breaking. But now I completely shifted to this.

1await page.evaluate(() => {
2  debugger;
3});

For more great debugging tips, I highly recommend this short article on Debugging Tips from Google.

Performance Automation

There's quite a buzz going on about using Puppeteer to automate web performance testing. I couldn't write this article without giving a shout out to Addy Osmani on the work developed on addyosmani/puppeteer-webperf, which I couldn't recommend more. Within the project README.md you'll find the most organized set of examples to tune your performance automation.

Browser Support

According to the official documentation, you can use Puppeteer with Firefox, with the caveat that you might encounter some issues since this capability is experimental at the time of this writing. You can specify which browser to run via puppeteer.launch options API that I've covered in this section.

What are your favorite bits of Puppeteer? What would you recommend me to learn next?

If you liked this article, consider sharing (tweeting) it to your followers.