This repository has been archived by the owner on Nov 17, 2023. It is now read-only.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[MXNET-1258]fix unittest for ROIAlign Operator #13609
[MXNET-1258]fix unittest for ROIAlign Operator #13609
Changes from 2 commits
1be6c50
aaa9d39
6f3b55d
d0ada04
afcfe76
b9e15a6
7079f9a
521eb5c
6ab9233
0c7dbe4
072b208
8698ae8
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not typecast
x_low
tonp.float32
here itself and remove the many typecasting statements below?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The type of x_low is
int
in the C implementation.In Python, the default type of float is
float64
, however the float type in the C implementation isfloat32
.If the types are not consistent, there will be calculation error.
So I use
T
to typecast some variables tonp.float32
to keep the type consistency between C and Python unittest.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you add a more descriptive error message
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! I will add more detail.