[Bug]: Running a test with the new tagging syntax causes an error during test run

[jhilber]

Version

1.43.0

Steps to reproduce

Issue:
When running a test that includes the new tagging syntax I receive the following error... "TypeError: fn is not a function". If I remove the tagging the test runs normal. Please find an example of the test syntax and the command line call used to run all tests.

Test example:

test(
'AT_101_01_Test',
{
tag: '@smoke',
},
async ({ testPage, testHelper }) => {
<< Test Code Here >>
}
}
);

Command line:
npx playwright test --project="ProjectApp" --workers=1

Expected behavior

Test to run and pass. This test rarely, if ever, fails.

Actual behavior

Fails with this error:

TypeError: fn is not a function

Additional context

Notice in the command line call I am not grepping for the tag. If I run the command line from a VS Code terminal window, I do not get the error and test runs fine. But if I run the same command line in a Powershell script in an Azure DevOps pipeline, I get the error.

Environment

System:
OS: Windows 10

Binaries:
Node: ^17.0.34
npm: 9.8.1

Node Packages:
"@playwright/test": "^1.43.0",

Language: Typescript

[mxschmitt]

Would it be possible to share a full reproduction starting from our getting started?

Most likely you still somewhere reference an old Playwright version. npm ls @playwright/test might yield some helpful output.

[jhilber]

@mxschmitt - The info in the Environment section above is what we should be using, but I will get the actual Azure DevOps pipeline output logging for you. Currently we do not output the versions after installing Playwright. I will have our Devops team add this to the pipeline task script following the install:

Write-Host "Installed Playwright: "(npx playwright --version)
Write-Host "Installed Node: "(node -v)
Write-Host "Installed NPM: "(npm -v)

Are there any other packages you would like for me to get? Unfortunately, I may not be able to get this to you for a couple of days as they are currently busy working on other projects.

FYI - We control the versions of the packages (including Playwright) from the package.json file. The version in the Environment section above is what we should be using based on that package file. I am assuming when we do a 'npm install' from the same directory where the package.json file is located, NPM and all package dependencies versions from that file are installed.

[yury-s]

@mxschmitt - The info in the Environment section above is what we should be using, but I will get the actual Azure DevOps pipeline output logging for you.

Is this problem reproducible for you locally or only on the ADO pipeline? There should be no difference in the behavior if you install using the same package.json in both environments.

[jhilber]

@yury-s - You are correct, there should not be any differences, and I will be able to confirm that once I get logging back indicating the versions installed. To answer your question... I am unable to reproduce locally in a VS Code terminal using the same PowerShell command call, using the same package.json file. It is reproducible in the pipeline every time though. As soon as I remove the tag, the test runs fine.

I was able to speak to a member of another team here that is using the same version of Playwright in their pipeline, and they are not having this problem at all. I looked at their package.json file and the only thing that stood out to me is they installed a Typescript package. And since this appears to be a Typescript error... maybe that could be the issue??? I am going to install Typescript from NPM and see if that makes a difference.

@mxschmitt - I was unable to get my change into ADO as our Devops team is pretty swamped at the moment, but as soon as I can get their time, I will get version logging from the pipeline.

[yury-s]

I am unable to reproduce locally in a VS Code terminal using the same PowerShell command call, using the same package.json file.

This is another indication that it has something to do with the version of playwright used/the environment in the ADO pipeline, we need to be able to reproduce this issue to act on it.

[jhilber]

I checked in another package.json file with a patch from 1.43.0 to 1.43.1. This morning the tagging worked just fine. I don't believe it is because the fix was in the patch. You all introduced the new tagging in 1.42 so if there were any issues in it... I imagine that 1.43.0 would have addressed them. One thing I did do though was installed the Typescript node package which I never installed before. I really don't believe this has any bearing on the issue either, but the error was a Typescript error, so I thought maybe we needed the latest Typescript package installed.

Ultimately, I think for whatever reason the package.json coming from source code did not have 1.43.0 in it as you both said. For now, we are working. I apologize for taking up your time, but much appreciate you all looking into it in a very prompt manner. This issue is not an issue and closed. Thank you.