MEGA: Make E2E Great Again #3444

Murderlon · 2022-01-21T12:14:20Z

After postponing this for a long time, here it is, a new e2e test suite focused on developer experience and ease of use.

Considered alternatives

Playwright instead of Cypress. Arguably a more modern take. It supports more browsers, has a node style API instead of the Cypress JQuery style API, and runs slightly faster. However, the docs are significantly less good. The main goal of this undertaking is to make e2e as easy and fast as possible. That means if you don't understand something, which is likely as e2e is complex, you want to find comfort in easy docs and best practices. Cypress offers this. It has guides, describes tradeoffs of testing strategies, and has real world examples. It is also more battle tested than Playwright and the error messages are great.
Vite instead of Parcel. I ran into trouble into getting Vite to play nice with the setup and ultimately tried Parcel, which worked out of the box. Other than that no strong opinion.

* main: (45 commits) Polished the latest update blog (#3390) docs: fix typo in audio.md (#3389) website: 2.0-2.3 post draft (#3370) Release: uppy@2.3.2 (#3383) meta: fix release script @uppy/core: document file.name (#3381) add `.npmignore` files to ignore .gitignore when packing (#3380) meta: add VSCode workspace settings to `.gitignore` Upgrade ws in companion (#3377) meta: use ESBuild to bundle in E2E test suite (#3375) meta: update linter config to parse ESM files (#3371) meta: move dev workspace to `private/` (#3368) meta: use Vite for examples/dev (#3361) website: remove dependency on `crypto` in @uppy/transloadit example (#3367) tools: enable linter on website examples (#3366) meta: enable linter on mjs scripts (#3364) Fix module field in @uppy/angular package.json (#3365) Release: uppy@2.3.1 (#3357) meta: improve release script wording and formatting meta: update npm deps (#3352) ...

* main: dev: move configuration to a `.env` file (#3430) Update ci.yml (#3428) @uppy/transloadit: simplify `#onTusError` (#3419) Force include babel numeric separator (#3422) Release: uppy@2.4.0 (#3420) @uppy/transloadit: ignore rate limiting errors when polling (#3418) @uppy/tus: pause all requests in response to server rate limiting (#3394) @uppy/transloadit: better defaults for rate limiting (#3414) Update BACKLOG.md Fix Companion deploys (#3388) update aws-presigned-url example to use esm (#3413) @uppy/image-editor: namespace input range css (#3406) Release: uppy@2.3.3 (#3410) improve private ip check (#3403) Add missing option to the screen capture types (#3400) @uppy/drag-drop: fix `undefined is not a function` TypeError (#3397) website: update december 2021 blog post (#3396)

* main: @uppy/transloadit: fix handling of Tus errors and rate limiting (#3429) Add Unsplash to website dashboard example (#3431)

* main: dev: fix Vite custom plugin (#3437) website: add legacy bundle to CDN example (#3433) meta: remove unused lerna and npm files (#3436) meta: replace browserify with esbuild (#3363) Release: uppy@2.4.1 (#3432)

.eslintrc.js

website/src/_template/contributing.md

bin/companion.sh

e2e/cypress/support/createFakeFile.ts

e2e/generate-test.mjs

aduh95

We should probably have a .env at the root that serves as an example / fallback for new contributors.

Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>

mifi

Looking very smooth sir! I left a few questions

mifi · 2022-02-03T12:38:02Z

bin/companion.sh

+if [ -f .env ]; then
+  # https://gist.github.com/mihow/9c7f559807069a03e302605691f85572?permalink_comment_id=3625310#gistcomment-3625310
+  set -a
+  source <(sed -e '/^#/d;/^\s*$/d' -e "s/'/'\\\''/g" -e "s/=\(.*\)/='\1'/g" .env)


Have you considered /~https://github.com/motdotla/dotenv ?

Yes, tried it here: 11ede08 but couldn't get it to work in CI.

What didn't work? didn't it load the .env file?
If .env was inside private/dev/.env, then I suspect that this line was the problem:

"start:companion": "yarn node private/dev/companion.js",

11ede08#diff-7ae45ad102eab3b6d7e7896acd08c427a9b25b346470d7bc6507b6481575d519R146

I think dotenv searches inside CWD and not inside the directory of the .js file, so for it to work it would have to be changed to:

"start:companion": "cd private/dev && yarn node companion.js",

e2e/generate-test.mjs

mifi · 2022-02-03T12:48:58Z

e2e/generate-test.mjs

@@ -0,0 +1,83 @@
+/* eslint-disable no-console, import/no-extraneous-dependencies */


I assume this script needs to be run for every new test to generate all the code/html needed for that test and that this generated code has to be checked into git. It would generate a lot of code that needs to be maintained in the future in case of changes. Could it be instead implemented as a function or module that is run for every test (kind of like a before hook) that will provide each test with the needed files on-the-fly?

We no longer need to generate files in order for tests to work like we used to. Now we simply import things and it will resolve to the yarn workspace. This is ran if you want the boilerplate of a new test, something that is ran once, not for every test.

Oh so this script needs to be run by the developer and commited to git for every new test the developer wants to write? Then I’m still wondering if there’s an alternative way, because it will lead to a lot ofduplication and boiler plate, if every new test needs this.

or are you saying this script is to be run only once in the lifetime of uppy codebase? If so, i’m not sure what’s the value of keeping such a script

I would suggest checking out the branch and running it to see what it does. To me it's valuable. But given that this wasn't immediately clear, I should perhaps update the docs better.

I understand it will generate files under client/ and a sample test file. I think what I'm not totally understanding yet, is how we are supposed to write new tests. If I or a contributor now want to add a test for a untested feature, like say upload a file from dropbox, should I then run generate-test.mjs to generate a new set of files for my test? And another feature, do I run it again? Or should I add an it statement into say /~https://github.com/transloadit/uppy/blob/956cd92eca9470661d948992278841f6d0f56cd9/e2e/cypress/integration/dashboard-transloadit.spec.ts and start adding my logic there? maybe we need to document this

e2e/generate-test.mjs

mifi · 2022-02-03T14:52:15Z

bin/companion.sh

+if [ -f .env ]; then
+  # https://gist.github.com/mihow/9c7f559807069a03e302605691f85572?permalink_comment_id=3625310#gistcomment-3625310
+  set -a
+  source <(sed -e '/^#/d;/^\s*$/d' -e "s/'/'\\\''/g" -e "s/=\(.*\)/='\1'/g" .env)


What didn't work? didn't it load the .env file?
If .env was inside private/dev/.env, then I suspect that this line was the problem:

"start:companion": "yarn node private/dev/companion.js",

11ede08#diff-7ae45ad102eab3b6d7e7896acd08c427a9b25b346470d7bc6507b6481575d519R146

I think dotenv searches inside CWD and not inside the directory of the .js file, so for it to work it would have to be changed to:

"start:companion": "cd private/dev && yarn node companion.js",

package.json

mifi · 2022-02-03T15:16:04Z

e2e/generate-test.mjs

@@ -0,0 +1,83 @@
+/* eslint-disable no-console, import/no-extraneous-dependencies */


I understand it will generate files under client/ and a sample test file. I think what I'm not totally understanding yet, is how we are supposed to write new tests. If I or a contributor now want to add a test for a untested feature, like say upload a file from dropbox, should I then run generate-test.mjs to generate a new set of files for my test? And another feature, do I run it again? Or should I add an it statement into say /~https://github.com/transloadit/uppy/blob/956cd92eca9470661d948992278841f6d0f56cd9/e2e/cypress/integration/dashboard-transloadit.spec.ts and start adding my logic there? maybe we need to document this

| Package | Version | Package | Version | | ------------------------- | ------- | ------------------------- | ------- | | @uppy/companion | 3.2.0 | @uppy/provider-views | 2.0.7 | | @uppy/companion-client | 2.0.5 | @uppy/thumbnail-generator | 2.1.0 | | @uppy/core | 2.1.5 | @uppy/robodog | 2.3.0 | | @uppy/dashboard | 2.1.4 | uppy | 2.5.0 | | @uppy/locales | 2.0.6 | | | - @uppy/companion: add support for COMPANION_UNSPLASH_SECRET (Mikael Finstad / #3463) - @uppy/unsplash: fix nested meta (Artur Paikin / #3485) - meta: fix(docs): typo in property `thumbnailType` (Dan Schalow / #3472) - @uppy/robodog: add audio, box, unsplash, screen-capture to Robodog (Artur Paikin / #3483) - meta: consolidate ENV files and fix contributing guidelines (Antoine du Hamel / #3475) - @uppy/companion-client,@uppy/companion,@uppy/provider-views,@uppy/robodog: Finishing touches on Companion dynamic Oauth (Renée Kooi / #2802) - meta: Improve companion docs (Mikael Finstad / #3479) - meta: Make E2E Great Again (Merlijn Vos / #3444) - meta: Add PostCSS handling to Vite (Artur Paikin / #3467) - meta: Update CONTRIBUTING.md (Mikael Finstad / #3411) - @uppy/companion: fix broken thumbnails for box and dropbox (Mikael Finstad / #3460) - website: fix `Uppy is not defined` error (Antoine du Hamel / #3461) - @uppy/companion: Implement periodic ping functionality (Mikael Finstad / #3246) - @uppy/companion: fix callback urls (Mikael Finstad / #3458) - @uppy/core,@uppy/dashboard,@uppy/thumbnail-generator: Add dashboard and UIPlugin types (Merlijn Vos / #3426) - @uppy/locales: Add "save" to fr_FR.js (Charly Billaud / #3395) - @uppy/companion: Fix TypeError when invalid initialization vector (Julian Gruber / #3416) - meta: Upgrade size-limit to 7.0.5 (Artur Paikin / #3445) - @uppy/provider-views: Unsplash: UI improvements (Artur Paikin / #3438) - @uppy/thumbnail-generator: exifr: remove legacy IE support (Artur Paikin / #3382) - @uppy/companion: Default to HEAD requests when the Companion looks to get meta information about a URL (Zack Bloom / #3417) - @uppy/dashboard: check if info array is empty (Artur Paikin / #3442) - meta: dev: fix Vite custom plugin (Antoine du Hamel / #3437) - website: add legacy bundle to CDN example (Antoine du Hamel / #3433) - meta: remove unused lerna and npm files (Antoine du Hamel / #3436) - meta: replace browserify with esbuild (Antoine du Hamel / #3363)

| Package | Version | Package | Version | | ------------------------- | ------- | ------------------------- | ------- | | @uppy/companion | 3.2.0 | @uppy/provider-views | 2.0.7 | | @uppy/companion-client | 2.0.5 | @uppy/thumbnail-generator | 2.1.0 | | @uppy/core | 2.1.5 | @uppy/robodog | 2.3.0 | | @uppy/dashboard | 2.1.4 | uppy | 2.5.0 | | @uppy/locales | 2.0.6 | | | - @uppy/companion: add support for COMPANION_UNSPLASH_SECRET (Mikael Finstad / transloadit#3463) - @uppy/unsplash: fix nested meta (Artur Paikin / transloadit#3485) - meta: fix(docs): typo in property `thumbnailType` (Dan Schalow / transloadit#3472) - @uppy/robodog: add audio, box, unsplash, screen-capture to Robodog (Artur Paikin / transloadit#3483) - meta: consolidate ENV files and fix contributing guidelines (Antoine du Hamel / transloadit#3475) - @uppy/companion-client,@uppy/companion,@uppy/provider-views,@uppy/robodog: Finishing touches on Companion dynamic Oauth (Renée Kooi / transloadit#2802) - meta: Improve companion docs (Mikael Finstad / transloadit#3479) - meta: Make E2E Great Again (Merlijn Vos / transloadit#3444) - meta: Add PostCSS handling to Vite (Artur Paikin / transloadit#3467) - meta: Update CONTRIBUTING.md (Mikael Finstad / transloadit#3411) - @uppy/companion: fix broken thumbnails for box and dropbox (Mikael Finstad / transloadit#3460) - website: fix `Uppy is not defined` error (Antoine du Hamel / transloadit#3461) - @uppy/companion: Implement periodic ping functionality (Mikael Finstad / transloadit#3246) - @uppy/companion: fix callback urls (Mikael Finstad / transloadit#3458) - @uppy/core,@uppy/dashboard,@uppy/thumbnail-generator: Add dashboard and UIPlugin types (Merlijn Vos / transloadit#3426) - @uppy/locales: Add "save" to fr_FR.js (Charly Billaud / transloadit#3395) - @uppy/companion: Fix TypeError when invalid initialization vector (Julian Gruber / transloadit#3416) - meta: Upgrade size-limit to 7.0.5 (Artur Paikin / transloadit#3445) - @uppy/provider-views: Unsplash: UI improvements (Artur Paikin / transloadit#3438) - @uppy/thumbnail-generator: exifr: remove legacy IE support (Artur Paikin / transloadit#3382) - @uppy/companion: Default to HEAD requests when the Companion looks to get meta information about a URL (Zack Bloom / transloadit#3417) - @uppy/dashboard: check if info array is empty (Artur Paikin / transloadit#3442) - meta: dev: fix Vite custom plugin (Antoine du Hamel / transloadit#3437) - website: add legacy bundle to CDN example (Antoine du Hamel / transloadit#3433) - meta: remove unused lerna and npm files (Antoine du Hamel / transloadit#3436) - meta: replace browserify with esbuild (Antoine du Hamel / transloadit#3363)

Murderlon added 14 commits November 22, 2021 16:04

attempt

a5d4bbd

attemps

6494bb7

More wip

4ff1745

another wip

4f529ea

fix typescript

7bfc7cd

Merge branch 'main' into cypress

c5386f6

* main: @uppy/transloadit: fix handling of Tus errors and rate limiting (#3429) Add Unsplash to website dashboard example (#3431)

Proper tests, generate test script, other improvements

39e2865

tweaks

2a0673a

Fix generate test formatting

bef8806

Fix eslint errors and add eslint-cypress

6e1f4a3

Merge branch 'main' into cypress

eb92b39

* main: dev: fix Vite custom plugin (#3437) website: add legacy bundle to CDN example (#3433) meta: remove unused lerna and npm files (#3436) meta: replace browserify with esbuild (#3363) Release: uppy@2.4.1 (#3432)

Add companion server, add more tests

e59a0c8

This comment has been minimized.

Sign in to view

Update contributing.md

77bfdad

aduh95 reviewed Jan 21, 2022

View reviewed changes

.eslintrc.js Outdated Show resolved Hide resolved

Murderlon and others added 13 commits January 21, 2022 17:23

Move .parceltc into e2e, fix env and companion setup

debcf10

Revert eslint formatting

422b874

Improve tus tests for reliability

47eff74

Update GitHub Actions

6d7cd80

Don't set companion url in env vars, always local

2134a09

fixup! Revert eslint formatting

cb5b04e

Update yarn.lock (fix yarn dev)

a165e96

remove unrelated changes (wrong merge?)

e56b84c

fixup! remove unrelated changes (wrong merge?)

88d3ce2

fixup! fixup! remove unrelated changes (wrong merge?)

169a622

Move all test resources into cypress

45d4d19

Add test for dashboard with react

a427bba

Create test for Vue 3 but disable it for now

29705ba

aduh95 reviewed Jan 31, 2022

View reviewed changes

website/src/_template/contributing.md Outdated Show resolved Hide resolved

Inject contributing.md in website and fix lint error

b66d7ec

aduh95 reviewed Feb 1, 2022

View reviewed changes

bin/companion.sh Outdated Show resolved Hide resolved

aduh95 reviewed Feb 1, 2022

View reviewed changes

e2e/cypress/support/createFakeFile.ts Show resolved Hide resolved

aduh95 reviewed Feb 1, 2022

View reviewed changes

e2e/generate-test.mjs Outdated Show resolved Hide resolved

aduh95 reviewed Feb 1, 2022

View reviewed changes

Murderlon and others added 3 commits February 2, 2022 12:12

Don't ignore .env.example

45c3158

Also do not forget to actually commit .env.example

7cce447

Apply suggestions from code review

c1b929b

Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>

mifi reviewed Feb 3, 2022

View reviewed changes

aduh95 reviewed Feb 3, 2022

View reviewed changes

e2e/generate-test.mjs Show resolved Hide resolved

Murderlon added 3 commits February 3, 2022 16:24

Make generate-test executable

2716cdd

Use top level await in generate-test

d31f09f

Fix lint errors

7ddf791

aduh95 approved these changes Feb 3, 2022

View reviewed changes

Actually fix lint error

956cd92

mifi reviewed Feb 3, 2022

View reviewed changes

aduh95 added 6 commits February 7, 2022 20:08

reuse VITE env variable for convenience

f460e7a

Upgrade npm deps

e542df4

fixup! reuse VITE env variable for convenience

69128fd

fix conflicts

7679bcb

Merge branch 'main' of /~https://github.com/transloadit/uppy

1699548

Update yarn.lock

ee9d715

aduh95 merged commit 6da874e into main Feb 8, 2022

aduh95 deleted the cypress branch February 8, 2022 16:37

mifi mentioned this pull request Feb 9, 2022

Issues after #3444 merged #3473

Closed

4 tasks

github-actions bot mentioned this pull request Feb 14, 2022

Release: uppy@2.5.0 #3486

Merged

aduh95 mentioned this pull request Feb 14, 2022

Release: uppy@2.5.0 #3487

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MEGA: Make E2E Great Again #3444

MEGA: Make E2E Great Again #3444

Murderlon commented Jan 21, 2022 •

edited

Loading

This comment has been minimized.

aduh95 left a comment

mifi left a comment

mifi Feb 3, 2022

Murderlon Feb 3, 2022

mifi Feb 3, 2022

mifi Feb 3, 2022

Murderlon Feb 3, 2022

mifi Feb 3, 2022

Murderlon Feb 3, 2022

mifi Feb 3, 2022

mifi Feb 3, 2022

mifi Feb 3, 2022

		@@ -0,0 +1,83 @@
		/* eslint-disable no-console, import/no-extraneous-dependencies */

MEGA: Make E2E Great Again #3444

MEGA: Make E2E Great Again #3444

Conversation

Murderlon commented Jan 21, 2022 • edited Loading

Considered alternatives

This comment has been minimized.

aduh95 left a comment

Choose a reason for hiding this comment

mifi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Murderlon commented Jan 21, 2022 •

edited

Loading