chore: lint system tests in CI #25673

flotwig · 2023-02-01T20:01:32Z

For cypress-io/cypress-internal#513

Additional details

This package was missing a lint command, so I added one
eafaf03 can be ignored during review, it's all auto-fix changes

Steps to test

How has the user experience changed?

PR Tasks

[na] Have tests been added/updated?
[na] Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
[na] Has a PR for user-facing changes been opened in cypress-documentation?
[na] Have API changes been updated in the type definitions?

revert jquery.js run ci a

cypress · 2023-02-01T21:02:30Z

9 flaky tests on run #44043 ↗︎

0	5103	74	0	9

Details:

Merge branch 'develop' into fix-system-tests-lint
Project: cypress	Commit: `12843047e3`
Status: Passed	Duration: 14:03 💡
Started: Feb 13, 2023 9:21 PM	Ended: Feb 13, 2023 9:36 PM

e2e/origin/cookie_behavior.cy.ts • 4 flaky tests • 5x-driver-electron

View Output Video

Test
... > same site / cross origin > XMLHttpRequest > sets cookie on same-site request if withCredentials is true, and attaches to same-site request if withCredentials is true
... > same site / cross origin > fetch > sets same-site cookies if "include" credentials option is specified from request, but does not attach same-site cookies to request by default (same-origin)
... > same site / cross origin > XMLHttpRequest > sets cookie on same-site request if withCredentials is true, and attaches to same-site request if withCredentials is true
... > same site / cross origin > fetch > sets same-site cookies if "include" credentials option is specified from request, but does not attach same-site cookies to request by default (same-origin)

commands/waiting.cy.js • 1 flaky test • 5x-driver-electron

View Output Video

Test
... > errors > throws waiting for the 3rd response

e2e/origin/commands/assertions.cy.ts • 1 flaky test • 5x-driver-electron

View Output Video

Test
cy.origin assertions > #consoleProps > .should() and .and()

cypress/cypress.cy.js • 3 flaky tests • 5x-driver-electron

View Output Video

Test
... > correctly returns currentRetry
... > correctly returns currentRetry
... > correctly returns currentRetry

_{This comment has been generated by cypress-bot as a result of this
project's GitHub integration settings.}

run ci

emilyrohrbough · 2023-02-07T17:04:40Z

system-tests/projects/e2e/cypress/e2e/proxying.cy.js

+      // to regenerate: `openssl dgst -sha384 -binary **/static/simple_obstructive_code.js | openssl base64 -A`
+      script.integrity = 'sha256-BLwfivLJLqn8sjt9PEEXBlw6lO+DSZzFZ0rP/SWai2o='


why did this change?

simple_obstructive_code.js was linted and so the integrity hash changed

also just realizing i made a slight typo here

Suggested change

// to regenerate: `openssl dgst -sha384 -binary **/static/simple_obstructive_code.js | openssl base64 -A`

script.integrity = 'sha256-BLwfivLJLqn8sjt9PEEXBlw6lO+DSZzFZ0rP/SWai2o='

// to regenerate: `openssl dgst -sha256 -binary **/static/simple_obstructive_code.js | openssl base64 -A`

script.integrity = 'sha256-BLwfivLJLqn8sjt9PEEXBlw6lO+DSZzFZ0rP/SWai2o='

emilyrohrbough · 2023-02-07T17:06:34Z

...jects/pristine/expected-cypress-js-component-create-react-app-v5/cypress/support/commands.js

@@ -22,4 +22,4 @@
 //
 //
 // -- This will overwrite an existing command --
-// Cypress.Commands.overwrite('visit', (originalFn, url, options) => { ... })


Can we just delete these placeholder files? bloats the projects if we don't need them at all. could be a follow up.

Yeah we totally could. My priority here was just getting these system-tests linted, not to clean out these files. They're not hurting anything so I don't plan on opening a follow-up PR to remove them.

emilyrohrbough · 2023-02-07T17:07:55Z

packages/app/cypress/e2e/runner/ct-framework-errors.cy.ts

@@ -74,7 +74,7 @@ reactVersions.forEach((reactVersion) => {
      })

      verify('error on mount', {
-        line: 6,


Why did these change?

the Errors.cy.jsx files and other Errors.* files were changed so the stack trace locations changed

emilyrohrbough · 2023-02-07T17:08:55Z

system-tests/.eslintrc.json

-  ]
+  ],
+  "rules": {
+    "no-console": "off"


I don't think we should be blindly disabling no-console. Are we using it that often we can't disable inline? What for if we are?

IMO it's ok to do here since these are all tests in this directory - there's no risk of a console log making it to prod. It's used pretty often in system-tests, 207 results in 69 files for console.

are these specifically in the mocha test files?or also in the mocked projects we are testing?

Both. For example, some of the projects do console.logs from the plugin so it can be captured in snapshots.

This rule catches mistakes fairly often for me in other test files, but I can see the case for turning it off in system-tests/. I'm on the fence, but probably fine.

BlueWinds · 2023-02-13T21:35:01Z

system-tests/.eslintrc.json

-  ]
+  ],
+  "rules": {
+    "no-console": "off"


This rule catches mistakes fairly often for me in other test files, but I can see the case for turning it off in system-tests/. I'm on the fence, but probably fine.

BlueWinds · 2023-02-13T21:36:32Z

system-tests/project-fixtures/angular/src/app/zonejs-mocha-only.cy.ts

-          .should('not.exist')
-      )
+      cy.wrap(Cypress.$(window.top!.document.body)).within(() => {
+        return cy


Don't need the return, even though it keeps it the same as before the curly braces were added. I assume this was done automatically.

* fix: update newProject ref when switching between organizations in SelectCloudProjectModal (#25730) * chore: debug page tooltip distance and artifact border (#25727) * misc: debug page tooltip distance and artifact border * add changelog entry * fix CT test * fix: Improve error handling around calls to `this.next` in middleware (#25702) * chore: update changelog validation example (#25742) * misc: improve debug loading text wrap responsiveness (#25703) * misc: Increase max failures in IATR badge to 99 (#25737) * chore: exclude collaborator issues/PRs from triage project (#25769) * feat: add --auto-cancel-after-failures flag (#25237) Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> Co-authored-by: Matt Schile <mschile@cypress.io> Co-authored-by: Ryan Pei <ryanppei@gmail.com> Co-authored-by: Emily Rohrbough <emilyrohrbough@yahoo.com> * chore: Update v8 snapshot cache (#25592) * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * Update update_v8_snapshot_cache.yml * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache * chore: updating v8 snapshot cache --------- Co-authored-by: cypress-bot[bot] <2f0651858c6e38e0+cypress-bot[bot]@users.noreply.github.com> Co-authored-by: Ryan Manuel <ryanm@cypress.io> Co-authored-by: cypress-bot[bot] <47117332+cypress-bot[bot]@users.noreply.github.com> * fix: implement new graphql fields for spec counts (#25757) Co-authored-by: Stokes Player <stokes@cypress.io> Co-authored-by: Mike Plummer <mike-plummer@users.noreply.github.com> * feat: Bundle cy.origin() dependencies at runtime (#25626) Co-authored-by: cypress-bot[bot] <2f0651858c6e38e0+cypress-bot[bot]@users.noreply.github.com> Co-authored-by: Ryan Manuel <ryanm@cypress.io> * chore: remove zenhub from release process (#25701) Co-authored-by: Matt Schile <mschile@cypress.io> * feat: add Cypress.Commands.overwriteQuery (#25674) * feat: add Cypress.Commands.overwriteQuery Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> Co-authored-by: Zach Bloomquist <git@chary.us> * fix: spawn child process with process.env in macOS arm64 (#25753) Co-authored-by: Matt Schile <mschile@cypress.io> Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> Co-authored-by: Zach Bloomquist <github@chary.us> * chore: lint system tests in CI (#25673) * fix: Suppress filesystem errors during glob search (#25774) * chore: issue with ts-loader missing in binary and problematic esbuild norewrite construct (#25797) * chore: update changelog linting (#25809) * docs(guides): add more detail to code-signing (#25794) Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> * chore: update workflows.yml to include the v8 snapshot update branch (#25784) Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com> * chore: internal request preflight (#25772) --------- Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: cypress-bot[bot] <2f0651858c6e38e0+cypress-bot[bot]@users.noreply.github.com> Co-authored-by: Ryan Manuel <ryanm@cypress.io> Co-authored-by: Matt Henkes <mjhenkes@gmail.com> Co-authored-by: Zach Bloomquist <git@chary.us> * chore: bump for 12.6.0 release (#25812) * chore: release @cypress/webpack-batteries-included-preprocessor-v2.4.0 [skip ci] * chore: release @cypress/webpack-preprocessor-v5.17.0 [skip ci] * test: skip flaky GitDataSource test (#25825) * chore: making our add-to-triage-board workflow reusable within the Cypress-io org (#25820) * chore: Making our add to triage workflow callable from other projects inside the Cypress-io org in Github * chore: updated cypress-example-kitchensink version (#25828) * fix: duplicate and expired cookies (#25761) * chore: add regression tests for duplicate cookies and bad expiry times * avoid prepending domain with dot for cookies that are set with the server side jar. This is to avoid the cookie being duplicated if it is set or overridden in a different context (request that can actually set the cookie or via document.domain) * feat: use cookie.toString() in the cookie patch to more accurately set cookies on the document, which should include other properties besides key=value * fix: add logic to handle expired cookies in the document.cookie patch, as well as in CDP * chore: build binary for cookie fixes for users to test * chore: change name of fixture to something more accurate * chore: comment why we are using the toughCookie toString method in the patch * [run ci] * chore: add changelog entry * [run ci] * fix: revert back to key=value when getting document.cookie as those are the only values are displayed (oversight on my end) * [run ci] * chore: make compatible with cypress.require * fix: add tests for hostOnly/non hostOnly cookies to make sure property gets sent up to automation client correctly. No longer need custom cookie prop to determine destination * [run ci] * fix: stale unit test * chore: adjust comments * [run ci] * fix: bad domain logic * [run ci] * chore: remove irrelevant comment * [run ci] * fix: adjust cookie login text to spec hostOnly cookie within the cookie patch. This should yield the same behavior as we are bound to same origin within the spec bridge * [run ci] * [run ci] * fix: allow for cookies on request of same key to take precedence over cookies in the jar, regardless of how many hierachy cookies exist in the jar * chore: fix cookie misc tests for cy.origin (dont run cy.origin) * [run ci] * chore: skip misc cookie tests in webkit as headless behavior doesn't clear cookies between tests correctly * Revert "fix: allow for cookies on request of same key to take precedence over cookies in the jar, regardless of how many hierachy cookies exist in the jar" This reverts commit 17de188. * [run ci] * chore: split changelog entry into two parts * chore: update logic to remove else statement and add comments * [run ci] * chore: readd windows snapshot branch in workflows * [run ci] * chore: fix workflows from bad merge * [run ci] * Revert "chore: split changelog entry into two parts" This reverts commit 4352ef5. * [run ci] * fix: Fix type definitions for cy.reload() (#25779) Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> * misc: Debug header updates (#25823) * fix: allow running tests outside Vite project root folder (#25801) * fix: allow running tests outside Vite project root folder * update snapshots * add changelog entry --------- Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com> * fix: mount component in [data-cy-root] (#25807) * fix(angular): mount component in [data-cy-root] * fix e2e test * add changelog entry * changelog [skip ci] * changelog --------- Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com> * chore: updating add to triage baord github action to use org secret (#25868) * chore: updating add to triage board github action to use org secret * chore: release @cypress/angular-v2.0.2 [skip ci] * chore: release @cypress/vite-dev-server-v5.0.3 [skip ci] * chore: Update v8 snapshot cache (#25822) Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com> Co-authored-by: Ryan Manuel <ryanm@cypress.io> * feat: support host only cookies (#25853) * feat: allow setCookie API to take a hostOnly option * chore: add jsdoc/typescript description to render to users * chore: add changelog entry * [run ci] * chore: fix types * chore: fix cookie login tests * chore: update e2e cookie system tests * [run ci] * chore: fix cookie command tests. localhost cookies are calculated as hostOnly, which is consistent with how cypress works today * chore: fix system tests for cookies. * [run ci] * chore: fix system tests * chore: skip hostOnly assertions in webkit (for now) * [run ci] * chore: add property definitions to setCookieOptions * [run ci] * chore: add comments to hostOnly prop in firefox when setting a cookie * fix(webpack-dev-server): touch component-index during onSpecsChange to avoid writing to app file (#25861) * testing: try disabling uTimesSync and see what happens * build binaries [run ci] * fix: touch component index file instead of browser.js * build binaries [run ci] * update test * fix test * add test for custom HTML file in config * use existing component index in webpack-dev-server unit tests --------- Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com> * chore: release @cypress/webpack-dev-server-v3.2.4 [skip ci] * chore: improve types for server automation cookie client (#25836) * chore: improve types for automation cookies * [run ci] * fix: the cookie_behavior tests by syncing cookies immediately if … (#25855) * fix: fix the cookie_behavior tests by syncing cookies immediately if the application is already stable * chore: add changelog entry * [run ci] * chore: address comments from code review * feat: Public API for CT Framework Definitions (#25780) * chore: rework component onboarding in launchpad (#25713) * chore: refactoring and types * rework source of frameworks * revert rename * fix tests * fix more tests * types * update code * use same public API internally * rename interfaces * rename * work on dev server api * fix types * fix test * attempt to support getDevServerConfig * tests * add function to define framework [skip ci] * rework a lot of types * fix test * update tests and types * refactor * revert changes * lint * fix test * revert * remove * add "community" label [skip ci] * refactor * types * lint * fix bug * update function name * address feedback * improve types with Pick * refactor using type guard * correct label --------- Co-authored-by: Zachary Williams <ZachJW34@gmail.com> * chore: typing error * feat: scan for 3rd party ct plugins (#25749) * chore: refactoring and types * rework source of frameworks * revert rename * fix tests * fix more tests * types * update code * use same public API internally * rename interfaces * rename * work on dev server api * fix types * fix test * attempt to support getDevServerConfig * tests * add function to define framework [skip ci] * rework a lot of types * fix test * update tests and types * refactor * revert changes * lint * fix test * revert * remove * add "community" label [skip ci] * refactor * types * lint * fix bug * update function name * address feedback * feat: scan for 3rd party ct plugins * add e2e test * unit tests [run ci] * tweak resolution * rebase, address comments * fix windows paths * remove .gitignore * fix test --------- Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com> * lint config * spacing * try fix race cond * fix import error * build binary * try update snapshot * try using require * support namespaced definitions (#25804) * remove category * add icon prop * support esm -> cjs compiled typescript * fix test * misc: add CTA footer to launchpad framework dropdown (#25831) * remove test project dependencies * rebase * windows * windows again * add changelog entry * changelog * revert workflow * remove worklfow --------- Co-authored-by: Zachary Williams <ZachJW34@gmail.com> Co-authored-by: Adam Stone-Lord <adams@cypress.io> * chore: release @cypress/webpack-dev-server-v3.3.0 [skip ci] * fix: Add missing error message when `req.continue` is used incorrectly (#25884) --------- Co-authored-by: Adam Stone-Lord <adams@cypress.io> Co-authored-by: Zachary Williams <ZachJW34@gmail.com> Co-authored-by: Mike Plummer <mike-plummer@users.noreply.github.com> Co-authored-by: Matt Schile <mschile@cypress.io> Co-authored-by: Alejandro Estrada <estrada9166@gmail.com> Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> Co-authored-by: Ryan Pei <ryanppei@gmail.com> Co-authored-by: Emily Rohrbough <emilyrohrbough@yahoo.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: cypress-bot[bot] <2f0651858c6e38e0+cypress-bot[bot]@users.noreply.github.com> Co-authored-by: Ryan Manuel <ryanm@cypress.io> Co-authored-by: cypress-bot[bot] <47117332+cypress-bot[bot]@users.noreply.github.com> Co-authored-by: Mark Noonan <mark@cypress.io> Co-authored-by: Stokes Player <stokes@cypress.io> Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com> Co-authored-by: Zach Bloomquist <git@chary.us> Co-authored-by: willmsC <50909991+willmsC@users.noreply.github.com> Co-authored-by: Zach Bloomquist <github@chary.us> Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com> Co-authored-by: Tim Griesser <tgriesser10@gmail.com> Co-authored-by: Matt Henkes <mjhenkes@gmail.com> Co-authored-by: semantic-release-bot <semantic-release-bot@martynus.net> Co-authored-by: Ben M <benm@cypress.io> Co-authored-by: Bill Glesias <bglesias@gmail.com> Co-authored-by: Podles <78863563+podlesny-j@users.noreply.github.com> Co-authored-by: Paolo Caleffi <p.caleffi@dreamonkey.com> Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com>

add lint command to system-tests

2dddd92

flotwig changed the title ~~style: enforce system test linting in CI~~ chore: enforce system test linting in CI Feb 1, 2023

flotwig force-pushed the fix-system-tests-lint branch from 3e9dc81 to 8394b11 Compare February 1, 2023 20:12

flotwig added 2 commits February 1, 2023 15:26

fix autofixable problems

212b3ed

revert jquery.js run ci a

manual fixes, ignore purposeful lint violations

38be77d

flotwig force-pushed the fix-system-tests-lint branch from 8394b11 to ce43de3 Compare February 1, 2023 20:28

flotwig added 4 commits February 2, 2023 13:22

revert todos fixtures changes

d521e0c

fix lintignore

18bfdb0

fix proxying.cy.js

9724886

run ci

05ce462

flotwig force-pushed the fix-system-tests-lint branch 4 times, most recently from 6abab65 to ca0b205 Compare February 2, 2023 19:54

fix pristine snapshots

31f2813

run ci

flotwig force-pushed the fix-system-tests-lint branch from 911386c to 31f2813 Compare February 3, 2023 18:32

flotwig added 2 commits February 6, 2023 13:45

Merge branch 'develop' into fix-system-tests-lint

2526efe

update stacktrace tests

5e619e2

run ci

flotwig changed the title ~~chore: enforce system test linting in CI~~ chore: lint system tests in CI Feb 6, 2023

flotwig marked this pull request as ready for review February 6, 2023 23:08

emilyrohrbough reviewed Feb 7, 2023

View reviewed changes

Update system-tests/projects/e2e/cypress/e2e/proxying.cy.js

aec4d1b

flotwig requested a review from emilyrohrbough February 9, 2023 16:38

emilyrohrbough approved these changes Feb 9, 2023

View reviewed changes

Merge branch 'develop' into fix-system-tests-lint

1284304

flotwig requested a review from BlueWinds February 13, 2023 21:29

BlueWinds approved these changes Feb 13, 2023

View reviewed changes

flotwig merged commit 748f3a5 into develop Feb 13, 2023

flotwig deleted the fix-system-tests-lint branch February 13, 2023 21:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: lint system tests in CI #25673

chore: lint system tests in CI #25673

flotwig commented Feb 1, 2023

cypress bot commented Feb 1, 2023 •

edited

Loading

emilyrohrbough Feb 7, 2023

flotwig Feb 7, 2023

flotwig Feb 7, 2023

emilyrohrbough Feb 7, 2023

flotwig Feb 7, 2023

emilyrohrbough Feb 7, 2023

flotwig Feb 7, 2023 •

edited

Loading

emilyrohrbough Feb 7, 2023

flotwig Feb 7, 2023

emilyrohrbough Feb 9, 2023

flotwig Feb 9, 2023

BlueWinds Feb 13, 2023

BlueWinds Feb 13, 2023

BlueWinds Feb 13, 2023

		// to regenerate: `openssl dgst -sha384 -binary **/static/simple_obstructive_code.js \| openssl base64 -A`
		script.integrity = 'sha256-BLwfivLJLqn8sjt9PEEXBlw6lO+DSZzFZ0rP/SWai2o='

chore: lint system tests in CI #25673

chore: lint system tests in CI #25673

Conversation

flotwig commented Feb 1, 2023

Additional details

Steps to test

How has the user experience changed?

PR Tasks

cypress bot commented Feb 1, 2023 • edited Loading

9 flaky tests on run #44043 ↗︎

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flotwig Feb 7, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cypress bot commented Feb 1, 2023 •

edited

Loading

flotwig Feb 7, 2023 •

edited

Loading