[JOSS Review] Cannot reproduce the results in the paper
Issue of reproducing the examples
I was following the installation instruction and installed the joss22
branch of sigcorr
based on this comment. When using the installed sigcorr
package to reproduce the results in the paper, I met a couple of issues:
- In the
Batched Statistics
example, the results ofvariance = bs.get_var()
is[0.6666667, 0.6666667]
instead of the one described in the paper ([3.3333333, 6.6666667]
). - In the
Statistical analysis of GP samples (parallel)
, it seemscov = np.eye(2.9) + 0.1
has a typo becausenumpy.eye
only accepts integers as the input arguments [ref]. I am using Python3.10 and numpy 1.24.3.
Please let me know if I missed anything during reproducing the results.
Tests failure on another machine
There is another issue that is not critical. When running tests on my local machine, there are several tests that fail. It seems everything works fine on the CI pipeline, but it might be worth noticing the precision differences across machines. The command line that I used to run tests is pytest tests/*
.
The test errors are
===================================== short test summary info ======================================
FAILED tests/test_gross_vitells_model.py::TestGrossVitells::test_gross_vitells_b_grad - AssertionError: 2.134941e-07 != 0 within 1e-07 delta (2.1349410417315084e-07 difference)
FAILED tests/test_gross_vitells_model.py::TestGrossVitells::test_gross_vitells_b_hess - AssertionError: 1.144131e-07 != 0 within 1e-07 delta (1.1441309766269114e-07 difference)
FAILED tests/test_gross_vitells_model.py::TestGrossVitells::test_gross_vitells_sb_hess - AssertionError: 1.3547442e-07 != 0 within 1e-07 delta (1.354744227910487e-07 difference)
FAILED tests/test_hyy_model.py::TestHyy::test_hyy_b_grad - AssertionError: 5.9093173e-07 != 0 within 1e-07 delta (5.909317337682296e-07 difference)
FAILED tests/test_hyy_model.py::TestHyy::test_hyy_sb_grad - AssertionError: 3.8397187e-07 != 0 within 1e-07 delta (3.8397186585825693e-07 difference)
FAILED tests/test_hyy_model.py::TestHyy::test_hyy_sb_hess - AssertionError: 1.6656135e-07 != 0 within 1e-07 delta (1.6656134960157942e-07 difference)
=================================== 6 failed, 5 passed in 11.79s ===================================
The specification of the machine is
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7K62 48-Core Processor
CPU family: 23
Model: 49
Thread(s) per core: 1
Core(s) per socket: 2
Socket(s): 1
Stepping: 0
BogoMIPS: 5190.24
LSB Version: core-11.1.0ubuntu4-noarch:security-11.1.0ubuntu4-noarch
Distributor ID: Ubuntu
Description: Ubuntu 22.04 LTS
Release: 22.04
Codename: jammy
Edited by Peifeng Jing