use fsspec.download for S3 artifacts + additional logging

[ebezzi]

Reviewers

Functional:
@bkmartinjr

Readability:

---

Changes

• Replaces the old method to download a remote h5ad with fsspec.download, which should use buffering instead of loading the whole file in memoy.
• When the S3 download function fails, no additional logs were displayed. Now they are (in verbose mode).

[bkmartinjr]

changes look OK and run fine on my laptop. Any idea why the unit & smoke tests fail?

[ebezzi]

changes look OK and run fine on my laptop. Any idea why the unit & smoke tests fail?

I believe those fail even in main (and in other recent PRs as well). I will look into them separately.

[codecov]

Codecov Report

Merging #2536 (4f548e7) into main (6c86216) will decrease coverage by 0.04%.
The diff coverage is 28.57%.

❗ Current head 4f548e7 differs from pull request most recent head e8a3229. Consider uploading reports for the commit e8a3229 to get more accurate results

@@            Coverage Diff             @@
##             main    #2536      +/-   ##
==========================================
- Coverage   71.58%   71.53%   -0.05%     
==========================================
  Files          94       94              
  Lines        6500     6503       +3     
  Branches      770      770              
==========================================
- Hits         4653     4652       -1     
- Misses       1771     1775       +4     
  Partials       76       76              

Flag	Coverage Δ
frontend	71.53% <28.57%> (-0.05%)	⬇️
javascript	71.53% <28.57%> (-0.05%)	⬇️
smokeTest	?
unitTest	71.53% <28.57%> (-0.05%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
server/data_anndata/anndata_adaptor.py	83.09% <0.00%> (-1.60%)	⬇️
server/common/utils/data_locator.py	69.66% <100.00%> (-0.34%)	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us.