The CS department maintains a collection of pre-built conda environments for general use, which include CUDA library-version specific environments for testing cuda code with different versions of the CUDA libraries. While one can always create one's own conda environment, doing so can consume a considerable amount of limited home directory space, and setting up such environments for GPU computing can be tricky due to multiple channels including package versions which can be incompatible once installed. There is literally no downside to using the pre-built environments other than you won't be able to add your own packages. If you have suggestions for packages which should be included in a particular environment or if you need a bespoke conda environment for a CS class or a customized personal-use environment which is too large for the storage made available to you on the departmental servers, please contact CS tech staff at help@cs.utexas.edu. If you're teaching a class using python tools, we highly recommend letting us set up a conda environment for the class to use in order to ensure access to appropriate or specific python package versions. Having everyone in the class using the same version of packages helps with debugging/grading.
Anaconda vs. miniconda -- what's the difference? Miniconda is a stripped down version of anaconda that includes only the minimum set of packages necessary to constitute a conda environment. There is almost never a reason to install anaconda, as it consumes a considerable amount of disk space and almost certainly includes packages you won't be using. Use miniconda instead, adding the packages you need using conda install. It's also best practice to never install packages to the base environment. Use conda create to create a new environment, conda activate to activate it, and install packages there. Use different conda environments for dissimilar projects requiring different sets of packages. This won't result in wasted space in the form of package duplication, as conda creates hard links to already installed packages rather than installing them again for a new environment. The problem with installing packages in the base environment is you can quickly start to run into package incompatibilities. The more isolated a package environment is, the more likely it is you will be able to install the newest versions of requisite packages with minimal conflict. For frequent users of conda, the Conda Cheatsheet is a helpful resource.
Conda works by manipulating your environment path so that the first executable found when you type python is the version in the environment. When you install anaconda or miniconda yourself, by default, changes are made to your .bashrc file resulting in a default conda environment being activated every time you open a new session. This is not necessarily what you want, so the CS conda environments are always inactive by default. You do not need to set any environment variables or modify your environment in any way to use the departmental conda environments.
Activating a CS conda environment
To activate a conda environment, type
source /lusr/opt/miniconda/bin/activate ENV_NAME
where ENV_NAME is the environment you want to activate. For example:
source /lusr/opt/miniconda/bin/activate numpy-numba
(numpy-numba is one of the environments you can activate. Currently available environments are listed below.)
Once any conda environment has been activated in a terminal, you can subsequently use the conda command:
conda activate numpy-numba
Deactivating a CS conda environment
To deactivate a currently active conda environment, type
conda deactivate
(Best practice is to only have one environment active at a time.)
To list all available conda environments (post initial activation):
conda env list
A selection of currently available environments
Most conda environments include many packages installed as dependencies. Generally these are not included in the lists below. Listed are actively installed packages; e.g. what you'd get by running "conda env export --from-history > MY_ENV.yml"
To see a complete list of available packages in a particular conda environment, activate the environment and type "conda list". To see the complete list of packages in an environment that is not activated, type "conda list -n ENV_NAME". There are often a surprising number of packages included as dependencies; e.g. the inclusion of the munkres package in the cupy-numba environment. It's useful and informative to glance over the list of packages when you first start using an environment you're not familiar with already.
Departmental conda environments are periodically updated. In order to allow for computational continuity, the last former version of any updated environment is retained under the same name with the update date as a suffix. So, for example, if you see two environments pytorch-cuda and pytorch-cuda_2024-04-17, this means the pytorch-cuda environment was updated on 2024-04-17. The new version is pytorch-cuda and the previous version is pytorch-cuda_2024-04-17. If you believe that a particular environment is in need of an update, contact help@cs.utexas.edu.
Note: environments with -cuda suffixes include packages compiled for use with Nvidia GPUs (AMD/ROCm versions to be built on demand). In many cases there is an environment with the same name save with a -cpu suffix. The -cpu version includes the same compute packages as the -cuda version, save compiled to run only on CPUs. All -cuda environments are self-contained and contain all necessary CUDA libraries. If you find that a library is missing from an environment, contact help@cs.utexas.edu
The following lists describe only a subset of the python packages included in these environments. To get a complete listing, activate conda and run conda list -n ENV_NAME where ENV_NAME is the name of the environment.
- cupy-numba
Includes:python 3.11.6CUDA 12.0cupy 12.2.0h5py 3.10.0jupyter notebookmatplotlib 3.8.0numba 0.57.1numpy 1.24.4pandas 2.1.1pillow 10.0.1polars 0.19.7scikit-learn 1.3.1scipy 1.11.3seaborn 0.13.0
- numpy-numba
Includes:python 3.11.5h5py 3.10.0jupyter notebookmatplotlib 3.8.0numba 0.57.1numpy 1.24.4pandas 2.1.1pillow 10.0.1polars 0.19.7scikit-learn 1.3.1scipy 1.11.3seaborn 0.13.0
- pycuda
Includes:python 3.11.6cudatoolkit 11.8.0h5py 3.9.0jupyter notebookmatplotlib 3.8.0numpy 1.26.0pandas 2.1.1pillow 10.0.1polars 0.19.7pycuda 2022.2.2scikit-learn 1.3.1scipy 1.11.3seaborn 0.13.0
- pytorch-cpu / pytorch-cuda
Includes:python 3.12.2CUDA 12.4 (-cuda version only)cudnn 8.9.7.29 (-cuda version only)h5py 3.10.0jupyter notebookmatplotlib 3.8.3numpy 1.26.4pandas 2.2.1pillow 10.2.0polars 0.20.18scikit-learn 1.4.1.post1scipy 1.12.0seaborn 0.13.0torchaudio 2.2.2torchvision 0.17.2validators 0.28.1
- tensorflow-cpu / tensorflow-cuda
Includes:python 3.11.8CUDA 12.4 (-cuda version only)cudnn 8.9.7.29 (-cuda version only)h5py 3.10.0jupyter notebookmatplotlib 3.8.3numpy 1.26.4pandas 2.2.1pillow 10.3.0polars 0.20.18scikit-learn 1.4.1.post12scipy 1.12.0seaborn 0.13.2
- cuda-dev-N.M
These are nvcc CUDA development environments available for testing and development. The N.M refers to the version of CUDA included. So for example, cuda-dev-11.8 includes CUDA version 11.8. All cuda-dev environments currently include the same version of CuDNN: 8.9.5.29, as this is the recommended version for all versions of CUDA 11.3--12.2. While a cuda-dev-12.2 is available, this isn't currently compatible with any GPUs in the CS compute clusters. The one you likely want to use for production work is cuda-dev-11.8
==== Conda environments for specific classes ====
- cs371n-cpu / cs371n-gpu
Includes:python 3.11.5CUDA 11.8.89 (GPU version)jupyter notebooknumpy 1.26.0nltk 3.8.1protobuf 4.24.4pytorch 2.1.0sentencepiece 0.1.99spacy 3.7.1torchaudio 2.1.0torchvision 0.16.0tqdm 4.66.1
