Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Companion: rewrite request and purest to got #3953

Merged
merged 9 commits into from
Aug 16, 2022
Merged

Conversation

mifi
Copy link
Contributor

@mifi mifi commented Aug 4, 2022

fixes #3496

TODO

  • provider: dropbox
  • provider: box
  • provider: drive
  • provider: onedrive
  • provider: facebook
  • provider: instagram
  • provider: unsplash
  • provider: zoom
  • fix jest
    • make jest work with ESM dynamic import, or
    • rewrite companion to ESM, or
    • downgrade got to non-ESM version
  • rewrite all usages of request
  • rewrite getRedirectEvaluator
  • rewrite form-data
  • rewrite custom HttpAgent / HttpsAgent
  • fix todos in code
  • see also Remove request and form-data from companion #3496

Breaking changes

  • Provider no longer gets config passed in its constructor object
  • /zoom/logout with invalid parameters will now return 400 instead of 500

mifi added 2 commits August 4, 2022 20:24
(not yet working due to jest esm issues)
@Murderlon Murderlon marked this pull request as draft August 4, 2022 18:37
@Murderlon Murderlon changed the title Draft: Rewrite request and purest to got Companion: rewrite request and purest to got Aug 4, 2022
@arturi
Copy link
Contributor

arturi commented Aug 4, 2022

Not required for Uppy 3.0, correct?

@Murderlon
Copy link
Member

Murderlon commented Aug 5, 2022

Not required for Uppy 3.0, correct?

Not unless we want to postpone the release by two weeks (has to be tested thoroughly). It's a breaking change unfortunately.

@mifi
Copy link
Contributor Author

mifi commented Aug 5, 2022

not required for companion 4, but this should be done sooner or later and will be a breaking change, so maybe companion 5

@kvz
Copy link
Member

kvz commented Aug 5, 2022

Our rough aim is no more than a major per year, so in that sense it could be either waiting two weeks or waiting 50. Yes we can deviate, but I still appreciate the reason behind this, which is to reduce friction and required attention of operating uppy on the consuming side. If 4 of us eat some discomfort so the ecosystem at large eats less of it and only has to pay attention to BREAKING notices ~once a year, that may be worth it?

@mifi
Copy link
Contributor Author

mifi commented Aug 8, 2022

after downgrading got to v11 (which is still CommonJS), the tests are going through, so I will continue with the port!

mifi added 5 commits August 9, 2022 00:54
- rewrite remaining providers to got
- rewrite to async/await
- pull out adapt code into adapters
- provider/companion tests still todo
- rewrite remaining providers to got and reuse code
- port tests
- remove request
- remove purest
- rewrite periodic ping job to got
- rewrite uploader to got
- rewrite "url" to got
- rewrite getRedirectEvaluator/request to got
- rewrite http/https agent/request to got
- rewrite credentials.js to got
- fix "todo: handle failures differently to return 400 for this case instead"
- add test for http/https agent
- improve test for credentials (remote/local)
- make /zoom/logout return 424 instead of 500 on credentials error
- remove useless http-agent tests
- fix various eslint warnings
@mifi mifi mentioned this pull request Aug 10, 2022
11 tasks
@mifi
Copy link
Contributor Author

mifi commented Aug 10, 2022

One remaining issue is that because we're running got v11, and it still depends on the old and unmaintained form-data that we wanted to get rid of. (see #3496) so in order to get rid of that we need to upgrade got to 12.

I have not actually manually tested:

  • OneDrive
  • Remote credentials (Transloadit)

I added some TODOs to #3284

@Murderlon Murderlon marked this pull request as ready for review August 10, 2022 10:11
@@ -87,7 +87,7 @@ export default () => {
// .use(Unsplash, { target: Dashboard, companionUrl: COMPANION_URL, companionAllowedHosts })
.use(RemoteSources, {
companionUrl: COMPANION_URL,
sources: ['Box', 'Dropbox', 'Facebook', 'GoogleDrive', 'Instagram', 'OneDrive', 'Unsplash', 'Url'],
sources: ['Box', 'Dropbox', 'Facebook', 'GoogleDrive', 'Instagram', 'OneDrive', 'Unsplash', 'Zoom', 'Url'],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't that be causing problems if devs don't have Zoom credentials in their .env?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not as long as you don't try to use the Zoom plugin right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, I use some of the others also without credentials, and it only fails once you try to use those providers

Comment on lines +30 to +38
async function withProviderErrorHandling ({ fn, tag, providerName, isAuthError, getJsonErrorMessage }) {
try {
return await fn()
} catch (err) {
const err2 = convertProviderError({ err, providerName, isAuthError, getJsonErrorMessage })
logger.error(err2, tag)
throw err2
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting pattern!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hah! probably not a textbook pattern but I wanted to reuse the error handling logic to make it consistent across all providers. maybe it could have been done with inheritance too

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alternatively we could pull the error handling logic out of the providers and instead do it in one place in the code that calls the provider methods

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inheritance is rarely the answer for me. I think it's alright as your basically doing inversion of control, and you're nudging towards going functional, as it's almost a Maybe monad, where you wrap a unpredictable action with flow control (stop or continue)

@mifi mifi merged commit 35812ca into main Aug 16, 2022
@mifi mifi deleted the request-purest-to-got branch August 16, 2022 11:44
@mifi mifi mentioned this pull request Aug 25, 2022
2 tasks
HeavenFox pushed a commit to docsend/uppy that referenced this pull request Jun 27, 2023
* rewrite to async

* rewrite box and dropbox to got

(not yet working due to jest esm issues)

* downgrade got

* update developer notes

* rewrite

- rewrite remaining providers to got
- rewrite to async/await
- pull out adapt code into adapters
- provider/companion tests still todo

* add zoom to dev dashboard

* rewrites

- rewrite remaining providers to got and reuse code
- port tests
- remove request
- remove purest
- rewrite periodic ping job to got
- rewrite uploader to got
- rewrite "url" to got
- rewrite getRedirectEvaluator/request to got
- rewrite http/https agent/request to got
- rewrite credentials.js to got
- fix "todo: handle failures differently to return 400 for this case instead"
- add test for http/https agent
- improve test for credentials (remote/local)
- make /zoom/logout return 424 instead of 500 on credentials error
- remove useless http-agent tests
- fix various eslint warnings

* work around ts error

* remove forgotten change
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove request and form-data from companion
5 participants