Activating Watson Machine Learning Community Edition on Euler
These instructions may also be used for generic Anaconda environments
Watson Machine Learning Community Edition (WML CE) is the successor to IBM’s PowerAI framework and is the recommended method for installing a number of ML packages on POWER (ppc64le) architectures. WML packages are also available for Intel and AMD CPUs, so Euler users will be able to utilize the same workflow for either architecture.
- Users must first run
module load anaconda/wml1 on the Slurm partition in which they want to use WML.
It is important that the these commands be run within a slurm job since the conda environment is always initialized only for the current architecture.
bootstrap_condain your shell. This is an alias for the commands necessary to activate conda without restarting your shell. Only bash, zsh, and fish shells are supported at this time, but others may be added upon request.
DO NOT use
conda initto install Anaconda to your shell’s startup files. Doing so will lock you in to a specific version and architecture for conda, and it does not work correctly inside batch jobs submitted via Slurm.
- Finally, activate the Watson Machine Learning Community Edition conda environment
conda activate wmlce. This will add all of the machine learning packages available on your current architecture to your environment.
If you wish to configure your own environment, you need only follow the directions for steps 1 and 2. Instructions for adding WML CE packages to your custom environment can be found on the IBM Website.
Using Microsoft PowerShell Core on Euler
Some users who are more familiar with Microsoft Windows may find that they are
more comfortable using PowerShell for scripting instead of the UNIX command
line. For these users, Euler provides access to PowerShell Core 6.2 for Linux
in the form of the
As of Sept 2019, the System Administrator does not recommend using PowerShell as
your default system shell. However, you may add the following snippet to
~/.bashrc file to launch it automatically on nodes where it is available.
if [ -z "$SLURM_JOB_ID" ]; then if [ -f /usr/bin/pwsh ]; then /usr/bin/pwsh exit $? fi fi
While it has not been tested extensively, PowerShell batch scripts may also be
used with Slurm by writing
#!/usr/bin/env pwsh as the first line of the script.
Users who require only a generic anaconda installation may use the module