Add and run CPMD regtests

This is an adaptation of the README file in [CPMD_ROOT_DIR]/regtests/cpqa.

When cloning the source code from the svn version, the lowest level directory should contain a sub-directory named regtests.

In the regtests folder or in a new directory, create a config.py file with the following settings:

# full path to CPMD root directory!
root='/home/pbaudin/Work/MTS_CPMD/cpmd_porting/CPMD'
arch='master'
version='x'
nproc=16
#
# path to executable
bin='${root}/bin_master/bin/cpmd.${version}'
#
# path to regtests
testsrc='${root}/../regtests/tests'
#
# path to PP library
ppdir='${root}/../regtests/tests/PP_LIBRARY/'
#
# Make command
make='make -j${nproc}'
#
# path to make directory
makedir='${root}/bin_master/'
#
# MPI details
nproc_mpi=16
mpi_prefix='mpirun -np %i'

Change the settings to suit your purposes.

CPQA first tries to compile the source code, we thus need to export the correct settings for the compilation (the Makefile and the $makedir folder should already exists, see Compile CPMD section for details). For example:

module load intel/15.0.3 intelmpi/5.1.1

To make the CPQA executable available we need to go inside the regtests/cpqa directory and run:

. loadenv.sh

The dot+space in front of loadenv.sh is mandatory. When opening a new terminal, the environment variables have to be loaded again.

Run the tests a first time with a version of CPMD you trust. This step generates the reference outputs. In the following example we limit the tests to the BO directory in ${testsrc}:

cpqa-main.py in/BO

The in directory will be created in the process. The reference outputs are stored in a directory with the prefix ref-. Each test run also creates a directory with prefix tst-. A symbolic link tst-*-last always points to the directory with the outputs of the last test run.

Add some feature or fix some bug. Then run the tests again. With git you can also change git branch and compare different branches against each other. In that case it is important to change branches in between test runs and eventually modify the config.py file:

cpqa-main.py in/BO

If things go wrong, you will notice error messages in the output.

After the execution of a test, the status of the test is marked by a series of flags. They have the following meaning:

D – DIFFERENT Some of the numbers in the output are different from the reference outputs.

E – ERROR The driver script encountered an internal error.

F – FAILED The CPMD binary returned a non-zero exit code.

L – LEAK A memory leak warning was detected in the standard error.

M – MISSING The output file could not be found.

N – NEW There is no reference output for this test.

O – OK All the tests for this input went fine. In the case of regression tests without reference outputs, the status is also OK.

R – RESET An additional RESET directive was found in the input. The new output is copied to the reference directory.

V – VERBOSE The CPMD binary writes some data to the standard output or standard error. This is not considered to be erroneous. However, when the test does not have the OK flag, the standard output and standard error is included in the final error report.

W – WRONG Some value in the output differs from the corresponding expected value.

Add a new regtest Simply create a test inside $testdir, e.g. copy and modify an existing one.

Then add the new test directory inside the file $testdir/TEST_DIRS.