Activating Watson Machine Learning Community Edition on Euler

Watson Machine Learning Community Edition (WML CE) is the successor to IBM’s PowerAI framework and is the primary method for installing a number of ML packages on POWER (ppc64le) architectures.

To access this environment, users must first run module load anaconda/3/2019.031 on the Slurm partition in which they want to use WML. It is important that this command be run within a job since the conda environment will always be initialized for the current architecture.

Execute powerai-init.sh2 to add the WML CE channels to your base conda environment. On Intel and AMD systems, this will cause the default versions of most ML frameworks to be overridden by IBM’s channels.

To install a specific Machine Learning package, follow the instructions for Installing Packages on IBM’s website.

Once the initial setup is complete, future jobs will need only run the appropriate module load anaconda/3/2019.03 and conda activate ... to access the installed packages.

Using Microsoft PowerShell Core on Euler

Some users who are more familiar with Microsoft Windows may find that they are more comfortable using PowerShell for scripting instead of the UNIX command line. For these users, Euler provides access to PowerShell Core 6.2 for Linux in the form of the pwsh binary.

As of Sept 2019, the System Administrator does not recommend using PowerShell as your default system shell. However, you may add the following snippet to your ~/.bashrc file to launch it automatically on nodes where it is available.

if [ -z "$SLURM_JOB_ID" ]; then
	if [ -f /usr/bin/pwsh ]; then
		exit $?

While it has not been tested extensively, PowerShell batch scripts may also be used with Slurm by writing #!/usr/bin/env pwsh as the first line of the script.

  1. Users who require Anaconda2 may use module load anaconda/2/2019.03 instead 

  2. This command is provided by Euler’s anaconda module, users do not need to provide a script.