lundi 29 décembre 2014

Why do the Theano tests fail with many "KnownFailureTest"s?

Theano is failing it's tests when I do:



python -c "import theano; theano.test();"


If these are known failures, shouldn't it still pass? IE when I test other libraries, KnownFailures sometimes trigger, but the overall test still passes with "OK" (but will still note the KnownFails and Skipped tests).


My guess is this is ok, and the test really is "passing", but since I'm doing a fresh install following the deeplearning.net tutorials, and I'm getting this error, I assume others might have this question as well, and a search on Google, and SO, isn't really helpful.


Forgive the error-code-dump, I am sure no one will need to read all through this, but it's here for reference if someone else has this question. Here are the errors at the end of the tests:



======================================================================
ERROR: test_none (theano.compile.tests.test_function_module.T_function)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/compile/tests/test_function_module.py", line 42, in test_none
raise KnownFailureTest('See #254: Using None as function output leads to [] return value')
KnownFailureTest: See #254: Using None as function output leads to [] return value

======================================================================
ERROR: test002_generator_one_scalar_output (theano.sandbox.scan_module.tests.test_scan.TestScan)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/sandbox/scan_module/tests/test_scan.py", line 474, in test002_generator_one_scalar_output
raise KnownFailureTest('Work-in-progress sandbox ScanOp is not fully '
KnownFailureTest: Work-in-progress sandbox ScanOp is not fully functional yet

======================================================================
ERROR: test003_one_sequence_one_output_and_weights (theano.sandbox.scan_module.tests.test_scan.TestScan)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/sandbox/scan_module/tests/test_scan.py", line 512, in test003_one_sequence_one_output_and_weights
raise KnownFailureTest('Work-in-progress sandbox ScanOp is not fully '
KnownFailureTest: Work-in-progress sandbox ScanOp is not fully functional yet

======================================================================
ERROR: test_alloc_inputs2 (theano.scan_module.tests.test_scan.T_Scan)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/tests/test_scan.py", line 2844, in test_alloc_inputs2
"This tests depends on an optimization for scan "
KnownFailureTest: This tests depends on an optimization for scan that has not been implemented yet.

======================================================================
ERROR: test_infershape_seq_shorter_nsteps (theano.scan_module.tests.test_scan.T_Scan)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/tests/test_scan.py", line 3040, in test_infershape_seq_shorter_nsteps
raise KnownFailureTest('This is a generic problem with infershape'
KnownFailureTest: This is a generic problem with infershape that has to be discussed and figured out

======================================================================
ERROR: test_outputs_info_not_typed (theano.scan_module.tests.test_scan.T_Scan)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: This test fails because not typed outputs_info are always gived the smallest dtype. There is no upcast of outputs_info in scan for now.

======================================================================
ERROR: test_arithmetic_cast (theano.tensor.tests.test_basic.test_arithmetic_cast)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/tests/test_basic.py", line 5583, in test_arithmetic_cast
raise KnownFailureTest('Known issue with '
KnownFailureTest: Known issue with numpy >= 1.6.x see #761

======================================================================
ERROR: test_abs_grad (theano.tensor.tests.test_complex.TestRealImag)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178

======================================================================
ERROR: test_complex_grads (theano.tensor.tests.test_complex.TestRealImag)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178

======================================================================
ERROR: test_mul_mixed (theano.tensor.tests.test_complex.TestRealImag)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178

======================================================================
ERROR: test_mul_mixed0 (theano.tensor.tests.test_complex.TestRealImag)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178

======================================================================
ERROR: test_mul_mixed1 (theano.tensor.tests.test_complex.TestRealImag)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178

======================================================================
ERROR: test_polar_grads (theano.tensor.tests.test_complex.TestRealImag)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178

======================================================================
ERROR: test_gradient (theano.tensor.tests.test_fourier.TestFourier)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178

======================================================================
ERROR: theano.tensor.tests.test_opt.test_log_add
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/tests/test_opt.py", line 1508, in test_log_add
raise KnownFailureTest(('log(add(exp)) is not stabilized when adding '
KnownFailureTest: log(add(exp)) is not stabilized when adding more than 2 elements, see #623

======================================================================
ERROR: Currently Theano enable the constant_folding optimization before stabilization optimization.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/tests/test_opt.py", line 3068, in test_constant_get_stabilized
"Theano optimizes constant before stabilization. "
KnownFailureTest: Theano optimizes constant before stabilization. This breaks stabilization optimization in some cases. See #504.

======================================================================
ERROR: test_dot (theano.tests.test_rop.test_RopLop)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/tests/test_rop.py", line 277, in test_dot
self.check_rop_lop(tensor.dot(self.x, W), self.in_shape)
File "/usr/local/lib/python2.7/dist-packages/theano/tests/test_rop.py", line 191, in check_rop_lop
raise KnownFailureTest("Rop doesn't handle non-differentiable "
KnownFailureTest: Rop doesn't handle non-differentiable inputs correctly. Bug exposed by fixing Add.grad method.

======================================================================
ERROR: test_elemwise0 (theano.tests.test_rop.test_RopLop)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/tests/test_rop.py", line 280, in test_elemwise0
self.check_rop_lop((self.x + 1) ** 2, self.in_shape)
File "/usr/local/lib/python2.7/dist-packages/theano/tests/test_rop.py", line 191, in check_rop_lop
raise KnownFailureTest("Rop doesn't handle non-differentiable "
KnownFailureTest: Rop doesn't handle non-differentiable inputs correctly. Bug exposed by fixing Add.grad method.

----------------------------------------------------------------------
Ran 2441 tests in 807.791s

FAILED (errors=18)


Thanks!


Aucun commentaire:

Enregistrer un commentaire