While our Python installations come with many popular packages installed, you may come upon a case where you need an addiditonal package that is not installed. If the specific package you are looking for is available from anaconda.org (formerlly binstar.org) you can easily install it and required dependencies by using the Conda package manager.
To be able to install a package using the Conda package manager:
- You must use an Anaconda distribution of Python:
- Load an appropriate python module, for example
module load python/3.6-conda5.2
on Pitzer. - Run
conda --version
to check that a version of Anaconda was successfully loaded.
- Load an appropriate python module, for example
- Package should be available through anaconda.org and/or using pip (see next section).
If you would like to freeze a distribution so that you can control how often you update Anaconda, please send us a help request at oschelp@osc.edu.
Procedure
The following steps are an example of how to set up a python environment and install packages to a local directory using Conda. We use the name local
for the environment, but you may use any other name.
Load proper python module
module load python/3.6-conda5.2
Create python installation to local directory
Three alternative create commands are listed. These cover the most common cases.
Clone base environment:
If you want to clone the full base python environment, you may use the following create command:
conda create -n local --clone base
Create New Environment:
The following will create a minimal python installation without any extraneous packages:
conda create -n local
Create New Environment with specific packages:
You can augment the command above by listing specific packages you would like installed into the environment. For example, the following will create a minimal python installation with only the specified packages (in this case, numpy
and babel
):
conda create -n local numpy babel
By default, conda will install the newest versions of the packages it can find. Specific versions can be specified by adding =<version>
after the package name. For example, the following will create a python installation with Python version 2.7 and NumPy version 1.16:
conda create -n local python=2.7 numpy=1.16
To verify that a clone has been created, use the command
conda info -e
For additional conda command documentation see https://docs.conda.io/projects/conda/en/latest/commands.html#conda-general-commands
Activate environment
Before the created environment can be used, it must be activated.
For the bash shell:
source activate local
On newer versions of Anaconda on the Owens cluster you may also need to perform the removal of the following packages before trying to install your specific packages:
conda remove conda-build
conda remove conda-env
Install packages
To install additional packages, use the conda install
command. For example, to install the yt
package:
conda install yt
By default, conda will install the newest version if the package that it can find. Specific versions can be specified by adding =<version>
after the package name. For example, to install version 1.16 of the NumPy package:
conda install numpy=1.16
Test python package
Now we will test our installed python package by loading it in python and checking its location to ensure we are using the correct version. For example, to test that NumPy is installed correctly, run
python -c "from __future__ import print_function; import numpy; print(numpy.__file__)"
and verify that the output generally matches
$HOME/.conda/envs/local/lib/python3.6/site-packages/numpy/__init__.py
To test installations of other packages, replace all instances of numpy
with the name of the package you installed.
Install your own python modules
If the method using conda above is not working or if you prefer, you can consider installing python modules from the source. Please read HOWTO: install your own python modules.
But I use virtualenv and/or pip!
See the comparison to these package management tools here:
https://docs.conda.io/projects/conda/en/latest/commands.html#conda-vs-pip-vs-virtualenv-commands
Pip installations are supported:
module load python module list # check which python you just loaded pip install --upgrade --user PACKAGE # where PACKAGE is a valid package name
If binaries exist in a pip installation they are usually installed in:
$HOME/.local/bin
The one issue with this approach is portability with multiple Python modules. If you plan to stick with a single Python module, then this should not be an issue. However, if you commonly switch between Python 2 and Python 3, then be aware of the potential trouble in using the same installation location for all Python versions.
Virtualenv may also work, but is not under active testing at OSC.
Further Reading:
Conda Test Drive: https://conda.io/docs/test-drive.html