Configuring a C++ environment on a server with conda

Preface Link to Preface

On a research group’s computing cluster, system-wide versions of C++ and MPI are often outdated for the sake of stability. This can cause compatibility issues during development.

While Conda is typically used for Python environments, it is also highly effective for configuring custom C++ environments, including specific versions of MPI and OpenMP.

Main Content Link to Main Content

Creating the Conda Environment Link to Creating the Conda Environment

Assuming Anaconda is already installed, run the following commands to create a new virtual environment with updated GCC and CMake:

1
2

conda create -n mpi_env gxx_linux-64=12.1.0 gcc_linux-64=12.1.0 cmake make -c conda-forge
conda activate mpi_env

Check if the compilers are correctly configured:

1
2

which gcc
which g++

Then, install MPI:

1

conda install mpich -c conda-forge

The mpich package includes the MPI libraries and essential tools like mpicc, mpicxx, and mpirun for compiling and running parallel programs.

⚠️ However, the system’s default mpicxx may still take precedence over the Conda-installed version. To ensure Conda’s version is prioritized when the environment is activated, run:

1

echo 'export PATH="$CONDA_PREFIX/bin:$PATH"' >> $CONDA_PREFIX/etc/conda/activate.d/force_conda_path.sh

Then, reactivate the environment:

1
2

conda deactivate
conda activate mpi_env

Verify the updated MPI toolchain:

1
2

which mpicxx
mpicxx -show

Now, you can install any additional C++ packages you need, such as Google Test:

1

conda install gtest -c conda-forge

To compile your C++ program:

1

mpicxx -std=c++11 -O2 -o mpi_demo mpi_demo.cpp

⚠️ Important: Conda’s version of libstdc++.so.6 may conflict with the system MPI or UCX libraries. Therefore, it’s recommended not to use srun to submit jobs, but instead use the mpirun provided by Conda’s mpich package. This ensures all processes use only the libraries inside the Conda environment.

Example SLURM Script Link to Example SLURM Script

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

#!/bin/bash
#SBATCH --account=hmt03
#SBATCH --job-name=mpi_demo
#SBATCH --partition=regular6430
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
#SBATCH --output=slurm-%j.out
#SBATCH --error=slurm-%j.err
#SBATCH --time=01:00:00    # ✅ Optional: set a max run time

# ✅ Initialize Conda for shell usage
source ~/Python/anaconda3/etc/profile.d/conda.sh

# ✅ Activate the desired environment
conda activate ALB_basis

# ✅ Force usage of libraries inside Conda
export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH

# ✅ Disable UCX and InfiniBand to avoid UCX-related errors
export UCX_TLS=^all
export OMPI_MCA_btl=^openib

# ✅ Use Conda's version of mpirun
$CONDA_PREFIX/bin/mpirun -n 2 ./mpi_demo