Caffe

Caffe is "a fast open framework for deep learning".

From their README:

Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by the Berkeley Vision and Learning Center (BVLC) and by community contributors. Yangqing Jia created the project during his PhD at UC Berkeley. Caffe is released under the BSD 2-Clause license.

Caffe also includes interfaces for both Python and Matlab, which have been built but have not been tested.

Availability and Restrictions

Versions

The following versions of Caffe are available on OSC clusters:

version Ruby Owens
1.0.0-rc3 X X

 

The current versions of Caffe on Ruby require the use of python/2.7.8, cuda/7.5.18, boost/1.56.0, opencv/3.1.0 on Ruby for GPU calculations.

The current version of Caffe on Owens requires cuda/8.0.44 for GPU calculations.

Feel free to contact OSC Help if you need other versions for your work.

Access 

Caffe is available to all OSC users without restriction.

Usage

Usage on Ruby or Owens

Setup on Ruby or Owens

To configure the Ruby or Owens cluster for the use of Caffe, use the following commands:

module load caffe

Batch Usage on Ruby or Owens

Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations for Ruby, and Scheduling Policies and Limits for more info.  In particular, Caffe should be run on a GPU-enabled compute node.

An Example of Using Caffe with MNIST Training Data on Ruby

Below is an example batch script ( job.txt ) for using Caffe, see this resource for a detailed explanation http://caffe.berkeleyvision.org/gathered/examples/mnist.html

#PBS -N Caffe
#PBS -l nodes=1:ppn=20:gpus=1:default
#PBS -l walltime=30:00
#PBS -j oe
#PBS -S /bin/bash 
cd $PBS_O_WORKDIR
. /etc/profile.d/lmod.sh
# Load the modules for Caffe
ml caffe
# Migrate to job temp directory and copy folders over
cd $TMPDIR
cp -r $CAFFE_HOME/{examples,build,data} .
# Download, create, train
./data/mnist/get_mnist.sh
./examples/mnist/create_mnist.sh
./examples/mnist/train_lenet.sh
# Serialize log files
echo; echo 'LOG 1'
cat convert_mnist_data.bin.$(hostname)*
echo; echo 'LOG 2'
cat caffe.INFO
echo; echo 'LOG 3'
cat convert_mnist_data.bin.INFO
echo; echo 'LOG 4'
cat caffe.$(hostname).*
cp examples/mnist/lenet_iter_10000.* $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the following command:

qsub job.txt
An Example of Using Caffe with MNIST Training Data on Owens

Below is an example batch script ( job.txt) for using Caffe, see this resource for a detailed explanation http://caffe.berkeleyvision.org/gathered/examples/mnist.html

#!/bin/bash
#PBS -N Caffe
#PBS -l nodes=1:ppn=28:gpu
#PBS -l walltime=30:00
#PBS -j oe
#PBS -S /bin/bash 
cd $PBS_O_WORKDIR
. /etc/profile.d/lmod.sh
# Load the modules for Caffe
ml caffe
# Migrate to job temp directory and copy folders over
cd $TMPDIR
cp -r $CAFFE_HOME/{examples,data} .
# Download, create, train
./data/mnist/get_mnist.sh
./examples/mnist/create_mnist.sh
./examples/mnist/train_lenet.sh
# Serialize log files
echo; echo 'LOG 1'
cat convert_mnist_data.bin.$(hostname)*
echo; echo 'LOG 2'
cat caffe.INFO
echo; echo 'LOG 3'
cat convert_mnist_data.bin.INFO
echo; echo 'LOG 4'
cat caffe.$(hostname).*
cp examples/mnist/lenet_iter_10000.* $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the following command:

qsub job.txt

Further Reading

http://caffe.berkeleyvision.org/

Service: 
Technologies: 
Fields of Science: