OSC is currently experiencing problems with its internal network. Interactive sessions may be slow or unresponsive, but running jobs should not be affected.
Users would encounter a MPI job failed with
openmpi/3.1.0-hpcx on Owens and Pitzer. The job would stop with the error like "There are not enough slots available in the system to satisfy the slots". Please switch to
openmpi/3.1.4-hpcx. The buggy version
openmpi/3.1.0-hpcx will be removed on August 18 2020.
Resolved: We removed
openmpi/3.1.0-hpcx on August 18 2020.
Users may experience unable to unload Intel software stack via module rm intel after switching between intel and ohter compilers. This is a known issue with current versions of Lmod on Pitzer and Ruby. The issue will be fixed by upgrading Lmod during system-wide downtime May 19th.
The CUDA debugger, cuda-gdb, can raise a segmentation fault immediately upon execution. A workaround before executing cuda-gdb is to unload the xalt module, e.g.:
module unload xalt
This issue affects most cuda modules on Pitzer and Owens.
Users may encoutner an error like 'libim_client.so: undefined reference to `uuid_unparse@UUID_1.0' while compiling MPI applications with mvapich2 in some Conda enivronments. We found pre-installed libuuid package from Conda conflicting with system libuuid libraries. The affected Conda packages are
Users may encounter under-performing MPI jobs or failures of compiling MPI applications if you are using Conda from system. We found pre-installed mpich2 package in some Conda environments overrides default MPI path. The affected Conda packages are
python/3.6-conda5.2. If users experience these issues, please re-load MPI module, e.g.
module load mvapich2/2.3.2 after setting up your Conda environment.
A batch job output may contain warnings about excessive memory usage.
We will have rolling reboots of Owens and Pitzer clusters including login and compute nodes, starting from Monday, February 3, 2020
We have found that large MPI jobs may hang at startup with
mvapich/2.3.1 (on any compiler dependency) due to a known bug that has been fixed in release 2.3.2. If users experience this issue, please switch to
NCBI blocks any connection from computing nodes because they are behind firewalls. Thus OSC users cannot use SRA tools to download data "on-the-fly" at runtime on computing nodes, e.g. 'fastq-dump -X 5 SRR390728'. OSC users must download SRA data on login using the command 'prefetch' before any sequence analysis. Please see the section 'Download SRA Data' in the SRA Toolkit software page for more detail.