Problems with LAMMPS 14May16

Category: 
Resolution: 
Resolved

LAMMPS 14May16 was built with the USER-OMP package on Oakley, Ruby, and Owens. Its default behavior is to spawn too many OpenMP threads. lammps/14May16 batch scripts that do not use the USER-OMP package should set the OMP_NUM_THREADS environment variable to 1 as a workaround, e.g.:
export OMP_NUM_THREADS=1
for Bourne type shells and
setenv OMP_NUM_THREADS 1
for C type shells.

Additional details:
The most obvious symptom is poor performance because the typical behavior is to spawn ppn times ppn OpenMP threads. For those using the USER-OMP package follow the advice in the manual carefully (section 5.3.4) and perform benchmarking to determine the best layout of MPI processes and OpenMP threads.

Resolution:

In fact, the USER-OMP package was not the source of the problem.  The source was building using -mkl which defaults to mkl=parallel.  The problem was fixed by rebuilding with mkl=sequential.

Since intentional usage of mkl=parallel was not expected by users, the corrected executables were made the default without advance user notification at these times:

Owens
Wed Nov 23 20:55:24 EST 2016

Ruby
Mon Nov 28 21:46:24 EST 2016

Oakley
Tue Nov 29 02:38:24 EST 2016

No user action is required; if a user had applied the OMP_NUM_THREADS workaround then it may be removed, but it will not cause probems if left in place.