Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PathNotFoundError in _load_convert for open_converted #447

Merged

Conversation

leewujung
Copy link
Member

open_converted failed on opening Zarr file without the beam_power group. The allowed exception GroupNotFoundError appears to only work for netCDF files but not with Zarr files:

PathNotFoundError: nothing found at path 'Beam_power'

Adding PathNotFoundError resolved the problem.

I also tested the case to only use PathNotFoundError without GroupNotFoundError and that works for both local netCDF and Zarr files.

From the zarr code here
/~https://github.com/zarr-developers/zarr-python/blob/master/zarr/errors.py
GroupNotFoundError is for when "group not found at path {0!r}"
and
PathNotFoundError is for when "nothing found at path {0!r}"

@imranmaj @lsetiawan : If I understand correctly right now the beam_power group is always created regardless of the sonar_model, but may be empty. So, there will be no case with exception GroupNotFoundError. Is this correct?

The current PR replaces GroupNotFoundError with PathNotFoundError.

@lsetiawan
Copy link
Member

If I understand correctly right now the beam_power group is always created regardless of the sonar_model, but may be empty

I don't remember if it's always created. Kavin did a lot of those group creation code, so I do not know. :/

@leewujung
Copy link
Member Author

I don't remember if it's always created. Kavin did a lot of those group creation code, so I do not know. :/

Right now on initialization EchoData this is done:

def __setup_groups(self):
for group in self.group_map.keys():
setattr(self, group, None)

which pulls all keys in group_map created from _get_convention
group_map = OrderedDict(_get_convention()["groups"])

I think the latter is you 😄 , but it doesn't matter.

Unless there's anything missing in this sequence, I am pretty sure this is correct.

@leewujung
Copy link
Member Author

leewujung commented Sep 29, 2021

Although, from the CI test failure, it seems that things may be handled differently on different platforms re what it means to have/no have a group, and whether there is anything in the path or group... I'll try adding GroupNotFoundError back in. If this fails, I'll have to properly set up the tests locally. 😛

@codecov-commenter
Copy link

codecov-commenter commented Sep 29, 2021

Codecov Report

Merging #447 (19b9c50) into dev (25fe601) will decrease coverage by 0.09%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##              dev     #447      +/-   ##
==========================================
- Coverage   76.74%   76.64%   -0.10%     
==========================================
  Files          38        5      -33     
  Lines        3324      334    -2990     
==========================================
- Hits         2551      256    -2295     
+ Misses        773       78     -695     
Flag Coverage Δ
unittests 76.64% <100.00%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
echopype/echodata/echodata.py 78.81% <100.00%> (-8.87%) ⬇️
echopype/preprocess/__init__.py
echopype/testing.py
echopype/convert/utils/ek_date_conversion.py
echopype/convert/parse_base.py
echopype/calibrate/__init__.py
echopype/calibrate/calibrate_azfp.py
echopype/__init__.py
echopype/calibrate/calibrate_base.py
... and 25 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 25fe601...19b9c50. Read the comment docs.

@leewujung leewujung added this to the 0.5.4 release milestone Sep 30, 2021
@leewujung leewujung merged commit ee9d35d into OSOceanAcoustics:dev Sep 30, 2021
leewujung added a commit that referenced this pull request Oct 1, 2021
* re-attach sonar_model to combined echodata object (#437)

* Correct duplicate ping_times during EK60 conversion (fixes #235) (#433)

* Add fix for duplicate ping_time

* Remove dead code

* Increment by 1ns instead of 1ms

* Add duplicate ping time warning and store in provenance

* Autoformatter

* Add test for duplicate ping times file

* Add qc method to remove duplicate pings

* Drop entire pings with duplicate values

* Store entire original ping_time and add attribute when there are duplicate ping_times

* Remove duplicate ping_time qc method, it will be added back in future PR

* Change EchoData combine ping_time reversal attribute to netCDF encodable value

* Remove unused import

* Update tests

* Clarify duplicate ping_time removal and drop warnings

* Add removal note to deprecation warnings (#443)

* Add removal note to deprecation warnings

* Say removal will be specifically in next release

* AD2CP conversion bug fixes (#438)

* Allow strings in AD2CP config string data record

* Add ping_time dimension to ahrs variables

* Default AHRS to empty list instead of None

* Change AHRS dims when it does not exist

* Add update_platform method to EchoData (#434)

* Add update_platform method to EchoData

* Add test for update_platform

* Use drop_vars instead of drop

* Use .data on variables

* Generalize extra_platform_data time dimension name

* Add AZFP support

* Generalized update_platform handling of CF trajectory-encoded datasets

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Default water_level to zeros instead of ones

* Add warning for overwritten variables

* Index incoming data by time2

* Add long_name for time2

* Show dropped variable warning when nans do not exist

* Reword dropped variable warning

* Rename mapping_get_multiple to mapping_search_variable

Co-authored-by: Emilio Mayorga <emiliomayorga@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Add summary statistics (#444)

* initial add of echo metrics functions

* update echo metrics to use range as data variable

* small readability edit

* added test functions to echopype.tests

* Modified syntax for consistency

* lint corrections on metrics

* added docstrings

* changed summary_statistics.py docstrings to numpy format

* Clean up tabs in code

* Clean up tabs in test code

* add metrics to run-test.py

* renamed test_metrics_summary_statistics

* small debug edits

* pre commit black edits

* moved test_metrics_summary_statistics.py to /tests/metrics/

* docstring update, add typing, change inertia to dispersion

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: stuara <stuara@uw.edu>
Co-authored-by: Landung "Don" Setiawan <landung.setiawan@gmail.com>
Co-authored-by: stuara <83487347+stuara@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Prepare workflows for master -> main (#445)

* Add PathNotFoundError in _load_convert for open_converted (#447)

* add PathNotFoundError in _load_convert for open_converted

Co-authored-by: imranmaj <49664304+imranmaj@users.noreply.github.com>
Co-authored-by: Emilio Mayorga <emiliomayorga@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: stuara <stuara@uw.edu>
Co-authored-by: Landung "Don" Setiawan <landung.setiawan@gmail.com>
Co-authored-by: stuara <83487347+stuara@users.noreply.github.com>
Co-authored-by: Landung "Don" Setiawan <landungs@uw.edu>
@leewujung leewujung deleted the open-converted-pathnotfounderror branch October 1, 2021 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants