Certain test have too stringent tolerances and fail unnecessarily (?)
This issue is related to the following email chain: https://listserv.fysik.dtu.dk/pipermail/gpaw-users/2016-June/004157.html
Currently, when running the tests on Mac OSX El cap, with gpaw freshly installed using pip, the following tests fail:
rasmusk at rasmus-macbook:/tmp/test2$ gpaw test -j 3
python-2.7.11 /usr/local/opt/python/bin/python2.7
gpaw-1.1.0 /Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/
ase-3.11.0 /usr/local/lib/python2.7/site-packages/ase/
numpy-1.11.1 /usr/local/lib/python2.7/site-packages/numpy/
scipy-0.17.1 /usr/local/lib/python2.7/site-packages/scipy/
_gpaw /Users/rasmusk/Library/Python/2.7/lib/python/site-packages/_gpaw.so
parallel /Users/rasmusk/Library/Python/2.7/bin/gpaw-python
FFTW no
scalapack no
libvdwxc no
PAW-datasets 1: /Users/rasmusk/gpaw-setups/gpaw-setups-0.9.11271
2: /usr/local/share/gpaw-setups
3: /usr/share/gpaw-setups
Running tests in /var/folders/23/3tv2l9s565v7pf_vzpczl6c80000gn/T/gpaw-test-bPH2Bb
Jobs: 3, Cores: 1, debug-mode: False
=============================================================================
solvation/vacuum.py 5.470 FAILED!
#############################################################################
Traceback (most recent call last):
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/__init__.py", line 577, in run_one
exec(compile(open(filename).read(), filename, 'exec'), loc)
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/solvation/vacuum.py", line 62, in <module>
equal(Etest, Eref, dE)
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/__init__.py", line 28, in equal
raise AssertionError(msg)
AssertionError: -11.9838020445 != -11.9837925246 (error: |-9.51991932929e-06| > 1e-09)
#############################################################################
inducedfield_lrtddft.py 7.839 FAILED!
#############################################################################
Traceback (most recent call last):
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/__init__.py", line 577, in run_one
exec(compile(open(filename).read(), filename, 'exec'), loc)
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/inducedfield_lrtddft.py", line 48, in <module>
equal(val1, 3175.76308, tol)
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/__init__.py", line 28, in equal
raise AssertionError(msg)
AssertionError: 3175.76177698 != 3175.76308 (error: |-0.00130302261596| > 0.0001)
#############################################################################
fdtd/ed_wrapper.py 23.614 FAILED!
#############################################################################
Traceback (most recent call last):
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/__init__.py", line 577, in run_one
exec(compile(open(filename).read(), filename, 'exec'), loc)
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/fdtd/ed_wrapper.py", line 43, in <module>
equal(qsfdtd.td_calc.hamiltonian.poisson.get_quantum_dipole_moment(), ref_qm_dipole_moment, tol)
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/__init__.py", line 28, in equal
raise AssertionError(msg)
AssertionError: [ 2.87661347e-10 1.99335501e-10 -9.35618521e-02] != [-2.63340461e-11, 2.61812794e-12, 0.521368171] (error: |[ 3.13995393e-10 1.96717373e-10 -6.14930023e-01]| > 0.0001)
#############################################################################
fdtd/ed.py 23.509 FAILED!
#############################################################################
Traceback (most recent call last):
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/__init__.py", line 577, in run_one
exec(compile(open(filename).read(), filename, 'exec'), loc)
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/fdtd/ed.py", line 83, in <module>
equal(td_calc2.hamiltonian.poisson.get_quantum_dipole_moment(), ref_qm_dipole_moment, tol)
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/__init__.py", line 28, in equal
raise AssertionError(msg)
AssertionError: [ 1.16182518e-09 -6.61938227e-11 -9.35619379e-02] != [-2.63340461e-11, 2.61812794e-12, 0.521368171] (error: |[ 1.18815922e-09 -6.88119506e-11 -6.14930109e-01]| > 0.0001)
#############################################################################
complex.py 6.133 FAILED!
#############################################################################
Traceback (most recent call last):
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/__init__.py", line 577, in run_one
exec(compile(open(filename).read(), filename, 'exec'), loc)
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/complex.py", line 23, in <module>
equal(Eini, Eini0, 1E-8)
File "/Users/rasmusk/Library/Python/2.7/lib/python/site-packages/gpaw/test/__init__.py", line 28, in equal
raise AssertionError(msg)
AssertionError: -17.8037677573 != -17.8037610364 (error: |-6.72086014575e-06| > 1e-08)
#############################################################################
For complex.py, this seems to be a matter of just reducing the tolerances to below 1e-6, but for some of the other tests, the size of the error is actually pretty large (e.g. 0.6 for the case of ed.py).
I will submit pull requests to alter the tolerances for complex.py, but perhaps we should worry about some of these tests?