Activating Watson Machine Learning Community Edition on Euler

These instructions may also be used for generic Anaconda environments

Watson Machine Learning Community Edition (WML CE) is the successor to IBM’s PowerAI framework and is the recommended method for installing a number of ML packages on POWER (ppc64le) architectures. WML packages are also available for Intel and AMD CPUs, so Euler users will be able to utilize the same workflow for either architecture.

  1. Users must first run module load anaconda/wml1 on the Slurm partition in which they want to use WML.
    • It is important that the these commands be run within a slurm job since the conda environment is always initialized only for the current architecture.

  2. Run bootstrap_conda in your shell. This is an alias for the commands necessary to activate conda without restarting your shell. Only bash, zsh, and fish shells are supported at this time, but others may be added upon request.
    • DO NOT use conda init to install Anaconda to your shell’s startup files. Doing so will lock you in to a specific version and architecture for conda, and it does not work correctly inside batch jobs submitted via Slurm.

  3. Finally, activate the Watson Machine Learning Community Edition conda environment with conda activate wmlce. This will add all of the machine learning packages available on your current architecture to your environment.

If you wish to configure your own environment, you need only follow the directions for steps 1 and 2. Instructions for adding WML CE packages to your custom environment can be found on the IBM Website.

Using Microsoft PowerShell Core on Euler

Some users who are more familiar with Microsoft Windows may find that they are more comfortable using PowerShell for scripting instead of the UNIX command line. For these users, Euler provides access to PowerShell Core 6.2 for Linux in the form of the pwsh binary.

As of Sept 2019, the System Administrator does not recommend using PowerShell as your default system shell. However, you may add the following snippet to your ~/.bashrc file to launch it automatically on nodes where it is available.

if [ -z "$SLURM_JOB_ID" ]; then
	if [ -f /usr/bin/pwsh ]; then
		exit $?

While it has not been tested extensively, PowerShell batch scripts may also be used with Slurm by writing #!/usr/bin/env pwsh as the first line of the script.

  1. Users who require only a generic anaconda installation may use the module anaconda/3 instead.