Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle exception while distributer not availble #219

Merged
merged 20 commits into from
Feb 15, 2024
Merged

Handle exception while distributer not availble #219

merged 20 commits into from
Feb 15, 2024

Conversation

shamlymajeed
Copy link
Contributor

Summary: closes #218

Related issue:

Suggested reviewer(s):

Reviewer checklist:

  • The headers of all files contain a reference to the repository license
  • 100% test coverage of new code - meaning:
    • The overall test coverage increased or remained the same as before
    • Every function is accompanied with a test suite
    • Tests are both positive (testing that the function work as intended with valid data) and negative (testing that the function behaves as expected with invalid data, e.g., that correct exceptions are thrown)
    • Functions with optional arguments have separate tests for all options
  • Examples are supported by doctests
  • All tests are passing
  • All names (e.g., files, classes, functions, variables) are explicit
  • Documentation (as docstrings) is complete and understandable

The checklist is based on the S-ENDA conventions and definition of done (see General Conventions). The above points are not necessarily relevant to all contributions. In that case, please add a short explanation to help the reviewer.

@shamlymajeed
Improve test codes
0102632 
 - verify whether the files are removed from workdir if any of the distributers fail or all distributers succeed.
@shamlymajeed
Handle exceptions if distrubuters fails to process or unavailable
d218abe 
 - Handled exceptions due to unavailabilty of distributers.
 - If any of the distributers is unavailable while distribution, api throws a 503 exception, so mmd-agent can retry sending that file later.
 - Fixed a test error.
@shamlymajeed
Handle unavailabity of distributer delete endpoint and improve test c…
bb772d9 
…overage.

 -Handled unavailabity of distributer in delete endpoint also.
 -Added test cases to test the exception handling while distributer is unavailable .
 -Improved test coverage.
@shamlymajeed shamlymajeed requested a review from mortenwh January 25, 2024 21:37
@shamlymajeed shamlymajeed self-assigned this Jan 25, 2024
mortenwh
mortenwh requested changes Jan 31, 2024
View reviewed changes
Copy link
Collaborator

@mortenwh mortenwh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to wok on the distribute function. See comments. Also, the errors seem to occur in dmci/distributors/distributor.py. As such, the error handling should be done there.

dmci/api/worker.py Outdated Show resolved Hide resolved
dmci/api/worker.py Outdated Show resolved Hide resolved
dmci/api/worker.py Outdated Show resolved Hide resolved
dmci/api/worker.py Outdated Show resolved Hide resolved
dmci/api/worker.py Outdated Show resolved Hide resolved
dmci/api/worker.py Outdated Show resolved Hide resolved
shamlymajeed and others added 2 commits February 1, 2024 15:08
@shamlymajeed @mortenwh
Update dmci/api/worker.py
fa4e991 
Co-authored-by: Morten W. Hansen <109061+mortenwh@users.noreply.github.com>
@shamlymajeed
Handle exceptions while any distributor is not available.
c3c30b8 
 -If any distributor is not available `status` becomes `False`, and moved to rejected.
 -improved test coverage.
@shamlymajeed shamlymajeed requested a review from mortenwh February 7, 2024 08:02
mortenwh
mortenwh requested changes Feb 7, 2024
View reviewed changes
Copy link
Collaborator

@mortenwh mortenwh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some changes needed..

dmci/api/app.py Outdated Show resolved Hide resolved
dmci/api/worker.py Outdated Show resolved Hide resolved
dmci/api/worker.py Outdated Show resolved Hide resolved
dmci/api/worker.py Outdated Show resolved Hide resolved
dmci/distributors/pycsw_dist.py Show resolved Hide resolved
tests/test_api/test_app.py Outdated Show resolved Hide resolved
tests/test_api/test_app.py Outdated Show resolved Hide resolved
tests/test_dist/test_pycsw_dist.py Outdated Show resolved Hide resolved
tests/test_dist/test_pycsw_dist.py Outdated Show resolved Hide resolved
tests/test_dist/test_pycsw_dist.py Outdated Show resolved Hide resolved
shamlymajeed and others added 10 commits February 7, 2024 11:39
@shamlymajeed @mortenwh
Update tests/test_api/test_app.py
7d4942d 
Co-authored-by: Morten W. Hansen <109061+mortenwh@users.noreply.github.com>
@shamlymajeed @mortenwh
Update tests/test_api/test_app.py
f61aefd 
Co-authored-by: Morten W. Hansen <109061+mortenwh@users.noreply.github.com>
@shamlymajeed @mortenwh
Update tests/test_dist/test_pycsw_dist.py
f256de0 
Co-authored-by: Morten W. Hansen <109061+mortenwh@users.noreply.github.com>
@shamlymajeed @mortenwh
Update tests/test_dist/test_pycsw_dist.py
8413716 
Co-authored-by: Morten W. Hansen <109061+mortenwh@users.noreply.github.com>
@shamlymajeed @mortenwh
Update tests/test_dist/test_pycsw_dist.py
e4b4153 
Co-authored-by: Morten W. Hansen <109061+mortenwh@users.noreply.github.com>
@shamlymajeed
Update pycsw_dist.py
e5ecede
@shamlymajeed @mortenwh
Apply suggestions from code review
9b81325 
Co-authored-by: Morten W. Hansen <109061+mortenwh@users.noreply.github.com>
@shamlymajeed
Update pycsw_dist.py
49a8429
@shamlymajeed
fixing flake error
3792f1c
@shamlymajeed
Fixed test failure
f7de61b
@shamlymajeed shamlymajeed requested a review from mortenwh February 7, 2024 12:27
mortenwh
mortenwh requested changes Feb 15, 2024
View reviewed changes
Copy link
Collaborator

@mortenwh mortenwh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small thing and then it's good - thanks!

dmci/distributors/pycsw_dist.py Outdated Show resolved Hide resolved
@mortenwh mortenwh merged commit 222655a into main Feb 15, 2024
7 checks passed
@mortenwh mortenwh deleted the Handle_exception_while_distributer_not_availble branch February 15, 2024 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mortenwh mortenwh mortenwh approved these changes

Assignees

@shamlymajeed shamlymajeed

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Handle the exception in dmci if any of the distibuters not available.
2 participants
@shamlymajeed @mortenwh