[Llvm-bgq-discuss] On compiling/running Gromacs with bgclang on BG/Q machine

[Jernej Zidar]

i there,

I've been trying to run the most recent version of Gromacs on a
Bluegene/Q machine. This meant I had to install bgclang (opted for the
stable release r284961) as a regular user, which was easier than
expected following the instructions here:
https://trac.alcf.anl.gov/projects/llvm-bgq

The I compiled my own cmake (3.4.1) and FFTW-3.3.6.

I compiled the source like this:
CMAKE_PREFIX_PATH=/home/ihpczidj/scratch/toolkit/fftw-3.3.6-gcc:/home/ihpczidj/scratch/toolkit/zlib-1.2.11-gcc/lib
\
cmake ../gromacs-2018.2 \
-DCMAKE_TOOLCHAIN_FILE=Platform/BlueGeneQ-static-bgclang-CXX
-DCMAKE_C_COMPILER=/home/ihpczidj/scratch/bgclang/mpi/bgclang/bin/mpicc
-DCMAKE_CXX_COMPILER=/home/ihpczidj/scratch/bgclang/mpi/bgclang/bin/mpicxx
-DGMX_MPI=ON -DGMX_BUILD_MDRUN_ONLY=ON -DGMX_DEFAULT_SUFFIX=OFF
-DGMX_BINARY_SUFFIX=-mpi-2018.2-bgq-bgclang
-DCMAKE_INSTALL_PREFIX=/home/ihpczidj/scratch/toolkit/ -DGMX_GPU=OFF
make
make install

There were no errors when compiling but when I try to run a short job
I would get this error:
mdrun-mpi-2018.2-bgq-bgclang: malloc.c:2357: sysmalloc: Assertion
`(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
__builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) ||
((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof
(struct malloc_chunk, fd_nextsize))+((4 * (sizeof(size_t))) - 1)) &
~((4 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) &&
((unsigned long)old_end & pagemask) == 0)' failed.

The above message would be repeated many times and followed by this:
*** Error in `/scratch/home/ihpc/ihpczidj/scratch/mem-bench/./mdrun-mpi-2018.2-bgq-bgclang':
malloc(): memory corruption: 0x0000001f80846c00 ***

What could be causing this error? Is it the compiler, Gromacs itself
or even my job file (currently: srun ./mdrun-mpi-2018.2-bgq-bgclang ... )?

Many thanks in advance,
Jernej