Supercomputing FAQ
General Questions
UNIX Questions
SSH Questions
Batch Processing Questions
Compiling System Questions
Parallel Processing Questions
Libraries/Software Questions
Performance Analysis Questions
Disk Storage
Other Common Problems
General Questions
Where should
a new OSC user begin?
The first thing you should do as a new OSC user is register your ARMSTRONG account using the login information in your welcome letter. You will use ARMSTRONG to change your HPC password, connect with other OSC users, add new users to your projects, and more. To register with your HPC information, click here.
After registering with ARMSTRONG and changing your password from the default provided, your next step will be configuring your computer to connect to the High Performance Computing (HPC) clusters at the OSC. For more information on connecting, read about the available computing environments and consult our FAQs on SSH.
Once you are able to connect to our HPC systems, you should start familiarizing yourself with the software and services available from the OSC, including:
What is an RU
(or mRU)?
An RU is a resource unit and an mRU
is a milli resource unit (i.e., 1 RU = 1000 mRUs). A resource unit is
an aggregate measure of the use of CPU, memory, and file storage. For
details on charging algorithms, see Charging.
How do I find
out my account balance?
To see usage and balance information
from any system, use the ARMSTRONG portal at https://armstrong.osc.edu/.
 |
accounting is updated once a
day, so the account balance is for the previous day. |
How do I
get/renew an account?
For information concerning accounts
(i.e., how to apply, who can apply, etc.), see Accounts.
How do I
register for a workshop?
Workshop announcements are posted
under Notices. For a complete schedule of of current training offierings, please visit the OSC Extended Training Schedule. To register or for more information, please
email trn-contact@osc.edu.
Where can I
find documentation?
For documentation specific to
software applications, see Software. For
other available documentation, see Manuals.
How do I read mail on OSC's supercomputing systems?
This service is not available at
this time.
How do I
install automatic mail forwarding from OSC's supercomputing systems?
This service is not available at this time. E-mail from OSC's supercomputing systems
is sent to the address we have for you in our database.
Can I
receive a letter of support from OSC when I apply for outside funding?
OSC has a standard letter of support
that you can include (electronically or in hard copy) with a proposal
for outside funding. This letter does not replace the application
process for time on OSC's systems. To receive the letter of support
please send your request to oschelp@osc.edu.
You should provide the following information: name and address of the
person/organization to whom the letter should be addressed; name(s) of
the principal investigator(s) and the institution(s); title of the
proposal; number of years of proposed account; number of RUs
requested per year. Please allow at least two working days to process your request.
When you receive outside funding related to research you conduct on OSC's systems, please log into the ARMSTRONG portal at https://armstrong.osc.edu/ and add that information in the appropriate section. Such information is important to the center and to state officials.
The letter includes the rates
associated for specified system(s). Hardware information about the
systems is available at http://www.osc.edu/supercomputing/hardware.
My
question isn't answered here. Whom can I ask for help?
OSC has a supercomputing support person
available for consultation during regular business hours (9am-5pm
Monday through Friday). More information on the OSC supercomputing help desk can
be found at
http://www.osc.edu/supercomputing/support/.
How do I acknowledge OSC in my publications?
An acknowledgment of support from the Ohio Supercomputer Center should appear in a publication of any material, whether copyrighted or not, based on or developed with OSC-supported computing resources.
"This work was supported in part by an allocation of computing time from the Ohio Supercomputer Center."
UNIX Questions
How do I change my password?
You can use the secure web page at
https://app.osc.edu/private/hpc-account.shtml to change your password. You may also log into the ARMSTRONG portal at https://armstrong.osc.edu/ and change your password in the appropriate section.
How do I change the default shell?
You can use the secure web page at https://app.osc.edu/private/hpc-account.shtml to change your shell. You may also log into the ARMSTRONG portal at https://armstrong.osc.edu/ and change your shell in the appropriate section.
How do I define my terminal type?
Appropriately set the TERM
environment variable. The most common terminal types are xterm
or vt100. The way you set the environment variable
depends on your shell. For example, a POSIX or Korn shell user may
enter export TERM=xterm to set the TERM variable
to xterm.
What does CPU time limit exceeded mean?
If you are running your application
interactively, you need to create a batch request. Your batch request
should include an appropriate estimate for the amount of CPU time that
your application will need. See Batch
Processing for more information.
How do I get more disk space?
Before you request additional space
for your permanent file system ,
consider temporarily using /tmp and/or compressing
files. For special requests, estimate the amount of disk space that you
will need and the duration that you will need it. Send special requests
to oschelp@osc.edu.
SSH Questions
What is SSH?
Secure Shell (SSH) is a program to
log into another computer over a network, to execute commands in a
remote machine, and to move files from one machine to another. It
provides strong authentication and secure communications over insecure
channels. SSH provides secure X connections and secure forwarding of
arbitrary TCP connections.
How can I upload or download files?
Since FTP is no longer supported at
OSC, you must use a utility that uses the SSH protocol. Current options
include Secure CoPy 'scp' and SSH File Transfer Protocol 'sftp'. These
utilities should be provided on most linux/unix platforms, but they can
also be found at the links given in the next section, "Where can I find
SSH clients".
For Windows users, a popular version of scp and SFTP exists called
WinSCP. This open source application can be obtained at no charge from:
http://winscp.sourceforge.net/eng
Where can I find SSH clients?
Windows users should consider using our OSC Starter Kit. This kit requires no configuration and includes shortcuts for initiating SSH connections with the HPC clusters, transferring files to and from the HPC file system, and links to helpful OSC websites. To find our more, click here.
For users interested in setting up their own environments, we recommend the following options:
- Windows
- Secure Shell, by SSH Communications Security. Free for non-commericial users.
- PuTTY by Simon Tatham. Open source.
- Linux and Mac OS X:
- Mac OS X and most Linux distributions come with the OpenSSH suite preinstalled, including the ssh and sftp command-line tools. For more information, open a terminal window and type "
man ssh" or "man sftp".
- Graphical SFTP clients:
How does SSH work?
SSH works by the exchange and
verification of information, using public and private keys, to identify
hosts and users. The ssh-keygen command
creates a directory ~/.ssh and files that
contain your authentication information, The public key is stored in ~/.ssh/identity.pub and the private key is stored
in ~/.ssh/identity. Share only your
public key. Never share your private key. To further protect
your private key you should enter a passphrase to encrypt the key when
it is stored in the file system. This will prevent people from using it
even if they gain access to your files.
One other important file is ~/.ssh/authorized_keys. Append your public
keys to the authorized_keys file and keep the same copy of it on each
system where you will make ssh connections.
How do I run a graphical application in an ssh session?
To do this, you need to be running
an X display server. On most Unix and Linux systems, you will probably
be running this already. If you are using a Mac running OSX, you will
need to install and run either the Apple
X11 server or XDarwin. On
Windows systems, there are numerous choices available, including X-Win32 and Cygwin/X. Most ssh
clients can be configured to automatically set up a remote X display as
part of the login session. This may happen without any action on your
part; if you log into an OSC machine and the command "echo $DISPLAY"
gives you more than a blank line, you need do nothing else. To
configure this automated behavior with OpenSSH on a Unix/Linux/OSX
system, you will need to either login in with the -X option or add the
following to $HOME/.ssh/config on your local machine:
Host *
ForwardX11 yes
ForwardX11Trusted yes
Note that some visualization
programs, such as AVS/Express, require an X server which supports the
GLX extension for OpenGL compatibility. To check if your X server
supports that extension, run the command "xdpyinfo
| grep GLX" in your login session to an OSC system. You
should get one or more lines of output if your X server supports this
extension.
Batch Processing Questions
What is a batch request?
On all OSC production systems, batch processing is managed by the Portable Batch System (PBS). PBS batch requests (jobs) are shell scripts that contain the same set of commands that you enter interactively.
These requests may also include options for the batch system that provide timing, memory, and processor information. For example
requests, see Batch Processing in the Supercomputing Environments section.
How do I submit, check the status, and/or delete a batch job?
PBS uses qsub to submit, qstat -a to check the status, and qdel to delete a batch request. For more information, see the online man pages.
Why won't my job run?
There are numerous reasons why a job might not run even though there appear to be processors and/or memory available. These include:
- Your account may be at or near the job count or processor count limit for an individual user.
- Your group/project may be at or near the job count or processor count limit for a group.
- The scheduler may be trying to free enough processors to run a large parallel job.
- Your job may need to run longer than the time left until the start of a scheduled downtime.
How can I retrieve files from unexpectedly terminated jobs?
A batch job that terminates before the script is completed can still copy files from $TMPDIR to the user's home directory via the trap command. In the batch script, the trap command needs to precede the command causing the TERMination. It could be placed immediately after the PBS header lines. Here is a generic form:
trap "cd $PBS_O_WORKDIR;mkdir $PBS_JOBID;cp -R $TMPDIR/* $PBS_JOBID" TERM
Compiling System Questions
What languages are available?
Fortran, C, and C++ are available on all OSC systems. The commands used to invoke the compilers and/or loaders vary from system to system. For more information, see "Using the System" under Supercomputing Environments.
When compiling an MPI2 C++ program I get the following
error: /usr/local/mpi/mvapich2-1.2p1-intel/include/mpicxx.h(37):
catastrophic error: #error directive: "SEEK_SET is #defined but must
not be for the C++ binding of MPI"
#error "SEEK_SET is #defined but must not be for the C++ binding of
MPI"
This error is the result of a problem in the C++ binding in
the MPI-2 standard. There is a namespace collision between MPI and
stdio, with SEEK_SET being defined in both. The recommend solution
is to use the compiler flag -DMPICH_IGNORE_CXX_SEEK.
A second option, although not recommended, is to include mpi.h before
stdio.h (works if you don't have nested includes).
Parallel Processing Questions
What is parallel processing?
The simultaneous use of more than one computer (or processor to solve a problem. There are many different kinds of parallel computers. They are distinguished by the kind of interconnection between processors or nodes (groups of processors) and between processors and memory.
What parallel processing environments are available?
On most systems, both shared-memory and distributed-memory parallel programming models can be used. Versions of OpenMP (for multithreading or shared-memory usage) and MPI (for message-passing or distributed-memory usage) are available. See Table 3 for a summary of parallel environments.
What is a core?
A core is a processor. When a single chip contains multiple processors, they are called cores.
What is Amdahl's law?
If F is the fraction of a calculation that is sequential, and (1-F) is the fraction that can be parallelized, then the maximum speedup that can be achieved by using P processors is 1/(F+(1-F)/P). This is the theoretical limit on speedup due to parallelization.
How do you do multi-thread using MVAPICH and OpenMP?
MVAPICH has a thread affinity option, which is supposed to
improved performance for single-threaded MPI processes. It prevents
the use of multiple cores in multithreaded (e.g., OpenMP) processes,
however. For MPI+OpenMP, set the environment variable
MV2_ENABLE_AFFINITY to 0:
bash:
export MV2_ENABLE_AFFINITY=0
csh:
setenv MV2_ENABLE_AFFINITY 0
Libraries/Software Questions
What software applications are available?
See Software section.
How do I get authorized to use a particular software application?
Access Agreement forms are available
on the Web in postscript and text formats. Where appropriate, they may
be found for a given application in the Software
section.
What math routines are available?
See the Software section for information on
third-party math libraries (e.g., ACML, fftw, scalapack, etc).
What scientific visualization software is available?
There are a number of visualization packages available, such as Gnuplot, Jmol, ParaView, and VTK.
The Visualization Toolkit, VTK, is an open source software system for 3D computer graphics, image processing and scientific visualization. A related application is ParaView. Paraview runs on distributed and shared memory parallel as well as single processor systems. Paraview uses VTK for the data
processing and rendering engine. The interface is written with a blend of TCL/TK and C++. Both VTK and Paraview were created by Kitware.
If another software package is needed, we have solutions available to run them remotely. Please contact oschelp@osc.edu for more information.
Also, see our training materials in the Training section.
OSC does not have a particular software package I would like to use. How can I request it?
Please refer to the general information of OSC's technical web pages entitled "Software". Under Forms, you will see the link to "Request for Software Form". Download the form, complete the information, and fax the form to OSC's software manager at 614/292-7168. The Statewide Users Group will consider the request.
What are modules?
Modules[1] are used to manage the environment variable settings associated with software packages in a shell-independent way. They work by setting up a special function or alias in your shell's environment called "module", which is used to evaluate commands and set environment variables in a transparent, reversible way. On OSC's systems, you will by default have modules in your environment for PBS, MPI, compilers, and a few other pieces of software. To see a list of your currently loaded modules, run:
module list
To see all of the modules available, run:
module avail
To add the module for a particular package to your environment, run:
module load package
Conversely, to remove a module from your environment, run:
module unload package
To see what changes a module makes to your environment, run:
module show package
[1] http://www.modules.org/
How do I convert files from an old Gaussian version to another version?
Almost all old Gaussian input files are valid for later versions.
However, the format for checkpoint files has been changed several times.
Using Gaussian09 for example,
the following command sequence performs a checkpoint conversion in place:
% cp -p mycalc.chk mycalc.chk.original
% module load g09
% c8609 mycalc.chk
The Gaussian09 module must be loaded to access the c8609 utility.
The mycalc.chk file will be rewritten in Gaussian09 format. For
reference, see http://www.gaussian.com/g_ur/m_utils.htm. The command c8609*.chk converts only the first file, so you must convert each checkpoint file separately. The conversion utility program name contains the version, so for the Gaussian03 module the program name is c8603.
Performance Analysis Questions
What are MFLOPS/GFLOPS/TFLOPS/PFLOPS?
MegaFLOPS/GigaFLOPS/TeraFLOPS/PetaFLOPS are millions/billions/trillions/quadrillions of FLoating-point (real number) Operations Per Second.
How do I find out about my code's performance?
A number of performance analysis tools (http://www.osc.edu/supercomputing/computing/index.shtml) are available on OSC systems. Some are general to all systems and others are specific to a particular system.
How can I optimize my code?
There are several ways to optimize code. Key areas to consider are CPU optimization, I/O optimization, memory optimization, and parallel optimization. Guidelines ( http://www.osc.edu/supercomputing/computing/opt-strategies.shtml ) are available to assist you.
Disk Storage
How can I determine the total disk space used by my account?
You can determine the total amount of disk space consumed by your
account by using the du command. From your home directory
issue the following command: du -sh .
Every OSC user shares their home file system with many other users. To
see how much disk space remains on your home file system you must first
identify the file system in which your files reside. To do this run
the command ls -l $HOME On the far right you will see
output similar to /home/osu0000 -> /h/osu0000/ This
example indicates that user osu0000 has their home
directory mounted on the /h file system. You can now use
this information to determine how much space remains on your home file
system by using the command df.
How can I find my largest directories?
To reveal the directories in your account that are taking up the
most disk space you can use the du, sort and
tail commands. For example, to display the ten largest
directories change to your home directory and then run the command:
du -k . | sort -n | tail -10
How can I find and delete core files?
Often when an application crashes or quits unexpectedly, an
image of that application's memory footprint at the time of the crash
is written to the file system as a "core file". These core files are
typically quite large and unless needed by the user should be deleted.
To find and delete these files go to your home directory and run the
command:
find . -name core\* -exec rm -f {} \;
How can I use tar and gzip to compress my files?
The commands tar and gzip can be used
together to produce compressed file archives representing entire
directory structures. These allow convenient packaging of entire
directory contents. For example, to package a directory structure
rooted at src/ use tar -czvf src.tar.gz src/
This archive can then be unpackaged using
tar -xzvf src.tar.gz where the resulting directory/file
structure is identical to what it was initially.
Are there other file compression programs?
The programs zip, bzip2 and
compress can also be used to create compressed file
archives. See the man pages on these programs for more details.
Other Common Problems
Why did my program die with a segmentation fault, address error, or signal 11?
This is most commonly caused by trying to access an array beyond its bounds -- for example, trying to access element 15 of an array with only 10 elements. Unallocated arrays and invalid pointers are other causes. You may wish to debug your program using one of the available tools such as TotalView.
I created a batch script in a text editor on a Windows or Mac system, but when I submit it on an OSC system, almost every line in the script gives an error. Why is that?
Windows and Mac have different end-of-line conventions for text files than UNIX and Linux systems do, and most UNIX shells (including the ones interpreting your batch script) don't like seeing the extra character that Windows appends to each line or the alternate character used by Macs. On most
OSC systems, you can use the following commands to convert a text file from Windows or Mac format to UNIX format:
dos2unix myfile.txt
mac2unix myfile.txt
|
