Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Additional unit test failures on setting execution engine to "naive" #15005

Open
juliusshufan opened this issue May 20, 2019 · 3 comments
Open
Labels
Backend Issues related to the backend of MXNet Test

Comments

@juliusshufan
Copy link
Contributor

juliusshufan commented May 20, 2019

With naïve engine (by export MXNET_ENGINE_TYPE = NaiveEngine)
Compared to the tests results using the default engine, there are additional UT failures, they are:

CASE NAME Error Log
test_quantization_mkldnn AssertionError: Items are not equal: Error 6065500000000000068157440.000000 exceeds tolerance rtol=0.000010, atol=0.000000.  Location of maximum error:(1, 12, 8, 6), a=121310.000000, b=0.000000
test_subgraph:test_pos_single_conv AssertionError: Items are not equal: Error 16186.005859 exceeds tolerance rtol=0.001000, atol=0.001000.  Location of maximum error:(3, 42, 0, 2), a=-16.600754, b=0.027311
test_subgraph:test_pos_conv_relu AssertionError: Items are not equal: Error 15091.447266 exceeds tolerance rtol=0.001000, atol=0.001000.  Location of maximum error:(2, 59, 3, 1), a=15.091448, b=0.000000
test_subgraph:test_pos_conv_bn AssertionError: Items are not equal: Error 83243.062500 exceeds tolerance rtol=0.001000, atol=0.001000.  Location of maximum error:(3, 53, 4, 7), a=152.526474, b=-0.842422
test_subgraph:test_pos_conv_add AssertionError: Items are not equal: Error 35976.011719 exceeds tolerance rtol=0.001000, atol=0.001000.  Location of maximum error:(3, 15, 7, 3), a=-41.533928, b=-0.150311
test_subgraph:test_pos_conv_add2 AssertionError: Items are not equal: Error 23006.433594 exceeds tolerance rtol=0.001000, atol=0.001000.  Location of maximum error:(0, 42, 6, 3), a=-23.886930, b=-0.036677
test_subgraph:test_pos_conv_bn_relu ssertionError: Items are not equal: Error 180822.453125 exceeds tolerance rtol=0.001000, atol=0.001000.  Location of maximum error:(0, 40, 3, 1), a=180.822464, b=0.000000
test_subgraph:test_pos_conv_bn_sum_relu AssertionError: Items are not equal: Error 167951.921875 exceeds tolerance rtol=0.001000, atol=0.001000.  Location of maximum error:(0, 58, 4, 0), a=167.951935, b=0.000000
test_exc_handling:test_exc_imperative mxnet.base.MXNetError: [10:45:55] src/operator/random/./sample_op.h:400: Check failed: param.scale > 0 (-1 vs. 0) : scale parameter in gaussian has to be positive
test_exc_handling:test_exc_gluon mxnet.base.MXNetError: [10:46:04] src/operator/random/./sample_op.h:400: Check failed: param.scale > 0 (-10 vs. 0) : scale parameter in gaussian has to be positive
test_exc_handling:test_exc_post_fail UnboundLocalError: local variable 'b' referenced before assignment
test_mkldnn:test_mkldnn_ndarray_slice TypeError: 'float' object is not subscriptable

@TaoLv @pengzhao-intel @eric-haibin-lin

@mxnet-label-bot
Copy link
Contributor

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Test

@juliusshufan juliusshufan changed the title Additional unit test failures on set execution engine to "naive" Additional unit test failures on setting execution engine to "naive" May 20, 2019
@vdantu
Copy link
Contributor

vdantu commented May 20, 2019

@mxnet-label-bot add [test]

@pengzhao-intel pengzhao-intel added the Backend Issues related to the backend of MXNet label May 31, 2019
@pengzhao-intel
Copy link
Contributor

@eric-haibin-lin @szha the MKLDNN related issue will be fixed in #15089.
Since people are using the naive engine for better performance now.
Does anyone in your side look into the engine?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Backend Issues related to the backend of MXNet Test
Projects
None yet
Development

No branches or pull requests

5 participants