[Feature]: The equivalent of cy.intercept, listening for requests, in order to use data from them

[OSnuszka]

🚀 Feature Request

When rewriting the playwright tests from cypress, the biggest problem I had was the lack of cy.intercept.
I currently have a workaround in playwright but it works very un-expectantly

export async function interceptResponse(searchString: string, page: Page) {
  const responsePromise = page.waitForResponse(
    (response) =>
      response.url().includes(searchString),
    { timeout: 60000 },
  );

  return responsePromise;
}

And example of use

let responsePromise = interceptResponse(`dataRequest.json`, page);
await page.goto(`/`);
let body = await (await responsePromise).text();
expect(responseBody).not.toBeNull();

I have more advanced uses of this function because I process json objects there, but the idea is the same -> intercept before the action that triggers the request -> action -> analyze data from the request.
I know about page.route -> but I don't know if I can use it in the context of listening and possibly returning data later on

Example

*The dataRequest.json appears during the page load

let request = page.intercept('dataRequest.json);
await page.goto('/');
await expect(request.body()).not.toBeNull();

Motivation

Easier process of rewriting automation from Cypress, more flexible automation

[yury-s]

I know about page.route -> but I don't know if I can use it in the context of listening and possibly returning data later on

Yes, you can. See this document: https://playwright.dev/docs/release-notes#response-interception You can fetch response, analyze it and then fulfill original request with it. Does this work for you?

[OSnuszka]

The example linked is a modification of the request, and in my case:

• I know that when the page loads there is a request that has JSON in the body. And after the page loads, it wants to use the data in JSON to look for elements on the page by selectors. (It does not want to do this asseration in the body of the page.route function).

[yury-s]

Can you explain then what is not working for your scenario in the following code?

    const responsePromise = page.waitForResponse('**/dataRequest.json');
    await page.goto('/');
    const response = await responsePromise;
    const data = await response.json();
    expect(data).toBeTruthy();
    // analyze the data and interact with the page

[OSnuszka]

Assertion is unstable because I am checking a request that has quite a large body - And playwright sometimes pulls it out too “fast” as the JSON is unprocessed.

[OSnuszka]

I rewrote the function that listens for requests and noticed that the tests work a little better (the ones that wait for requests). But I have a feeling that I'm not doing it optimally.

Just as I described waitForRequest itself, it doesn't wait for the body to be processed to the end in my case.

export async function interceptResponse(searchString: string, page: Page) {
  const response = page.waitForResponse(
    (response) =>
      response.url().includes(searchString) &&
      response.status() === 200 &&
      response.body() !== null &&
      response.request().failure() === null,
    { timeout: 30000 },
  );
  return response;
}

[mxschmitt]

And playwright sometimes pulls it out too “fast” as the JSON is unprocessed.

could you elaborate on that?

We need a reproduction which we can run locally in order to act on issues like that. Most likely there is a race somewhere, so looking at code is essential for us. Would it be possible to share a small test, ideally with expected and actual outcome?

[OSnuszka]

It's hard for me to find an adequate JSON example(Our JSON loads within half a second as part of the page opening.), and unfortunately, I can't share a real example, but it is very similar to the example below.

import { test, expect } from '@playwright/test';

for (let i = 0; i < 10; i++) {
test(`demo JSON + ${ i}`, async ({ page }) => {
  let promise = page.waitForResponse(
    (response) =>
      response.url().includes('feed_content?') &&
      response.status() === 200 &&
      response.body() !== null &&
      response.request().failure() === null,
    { timeout: 30000 },
  );

  await page.goto('https://dev.to/playwright');
  const response = await promise;
  const responseBody = JSON.parse(await response.text());
  let totalCount = responseBody["result"].length;
  expect(totalCount).toBeGreaterThan(0);
});
}

I looped the test to try to induce instability described as a topic of this task, but it works stably here.
Test in my case is failing at const responseBody = JSON.parse(await response.text());.

Once I try review trace (in case of test failing due instability) with to check request body in network tab I can see:

This test passes completely randomly, depending on whether the body manages to generate in time

[mxschmitt]

Usually this error happens when you try to access a response body of a request which was from a previous navigation. Browsers like Chromium will clean them up from their memory in order to save memory and give you this error. What might work is the following:

import { test, expect } from '@playwright/test';

for (let i = 0; i < 10; i++) {
  test(`demo JSON + ${i}`, async ({ page }) => {
    let resolveResponse = null;
    const waitForSpecificResponse = new Promise(resolve => resolveResponse = resolve);
    page.on('response', async response => {
      if (response.url().includes('feed_content?'))
        resolveResponse(await response.json());
    });
    await page.goto('https://dev.to/playwright');
    const responseBody = await waitForSpecificResponse;
    const totalCount = responseBody['result'].length;
    expect(totalCount).toBeGreaterThan(0);
  });
}

Alternatively try to delay the second navigation if possible.

[OSnuszka]

This solution works, but requires some refactoring on the side of my tests. As if there would be a dedicated function which one handle request in expected way in playwright for this I would be happy.

[yury-s]

Closing as the problem has been resolved and there are no planned changes on playwright side.