Neural Networks with TensorFlow¶
TensorFlow is a free end-to-end open-source software library for data flow and differentiable programming across many tasks. It is a symbolic math library, used primarily for machine learning applications. It has a comprehensive, flexible ecosystem of tools, libraries and community resources.
Please check the software modules list via
marie@compute$ module spider TensorFlow [...]
to find out, which TensorFlow modules are available on your partition.
On ZIH systems, TensorFlow 2 is the default module version. For compatibility hints between TensorFlow 2 and TensorFlow 1, see the corresponding section below.
We recommend using partitions
ml when working with machine learning workflows
and the TensorFlow library. You can find detailed hardware specification in our
On the partition
alpha, load the module environment:
marie@alpha$ module load modenv/scs5
Alternatively you can use
modenv/hiera module environment, where the newest versions are
marie@alpha$ module load modenv/hiera GCC/10.2.0 CUDA/11.1.1 OpenMPI/4.0.5 The following have been reloaded with a version change: 1) modenv/scs5 => modenv/hiera Module GCC/10.2.0, CUDA/11.1.1, OpenMPI/4.0.5 and 15 dependencies loaded. marie@alpha$ module avail TensorFlow -------------- /sw/modules/hiera/all/MPI/GCC-CUDA/10.2.0-11.1.1/OpenMPI/4.0.5 ------------------- Horovod/0.21.1-TensorFlow-2.4.1 TensorFlow/2.4.1 [...]
On the partition
ml load the module environment:
marie@ml$ module load modenv/ml The following have been reloaded with a version change: 1) modenv/scs5 => modenv/ml
This example shows how to install and start working with TensorFlow using the modules system.
marie@ml$ module load TensorFlow Module TensorFlow/2.3.1-fosscuda-2019b-Python-3.7.4 and 47 dependencies loaded.
Now we can use TensorFlow. Nevertheless when working with Python in an interactive job, we recommend to use a virtual environment. In the following example, we create a python virtual environment and import TensorFlow:
marie@ml$ ws_allocate -F scratch python_virtual_environment 1 Info: creating workspace. /scratch/ws/1/python_virtual_environment [...] marie@ml$ which python #check which python are you using /sw/installed/Python/3.7.2-GCCcore-8.2.0 marie@ml$ virtualenv --system-site-packages /scratch/ws/1/marie-python_virtual_environment/env [...] marie@ml$ source /scratch/ws/1/marie-python_virtual_environment/env/bin/activate marie@ml$ python -c "import tensorflow as tf; print(tf.__version__)" [...] 2.3.1
TensorFlow in JupyterHub¶
In addition to interactive and batch jobs, it is possible to work with TensorFlow using JupyterHub. The production and test environments of JupyterHub contain Python and R kernels, that both come with TensorFlow support. However, you can specify the TensorFlow version when spawning the notebook by pre-loading a specific TensorFlow module:
You can also define your own Jupyter kernel for more specific tasks. Please read about Jupyter kernels and virtual environments in our JupyterHub documentation.
TensorFlow in Containers¶
Another option to use TensorFlow are containers. In the HPC domain, the Singularity container system is a widely used tool. In the following example, we use the tensorflow-test in a Singularity container:
marie@ml$ singularity shell --nv /scratch/singularity/powerai-1.5.3-all-ubuntu16.04-py3.img Singularity>$ export PATH=/opt/anaconda3/bin:$PATH Singularity>$ source activate /opt/anaconda3 #activate conda environment (base) Singularity>$ . /opt/DL/tensorflow/bin/tensorflow-activate (base) Singularity>$ tensorflow-test Basic test of tensorflow - A Hello World!!!... [...]
In the above example, we create a conda virtual environment. To use conda, it is be necessary to configure your shell as described in Python virtual environments
TensorFlow with Python or R¶
For details on how to run TensorFlow with multiple GPUs and/or multiple nodes, see distributed training.
Compatibility TF2 and TF1¶
TensorFlow 2.0 includes many API changes, such as reordering arguments, renaming symbols, and
changing default values for parameters. Thus in some cases, it makes code written for the TensorFlow
1.X not compatible with TensorFlow 2.X. However, If you are using the high-level APIs (
there may be little or no action you need to take to make your code fully
TensorFlow 2.0 compatible. It is still possible to
run 1.X code, unmodified (except for
contrib), in TensorFlow 2.0:
import tensorflow.compat.v1 as tf tf.disable_v2_behavior() #instead of "import tensorflow as tf"
To make the transition to TensorFlow 2.0 as seamless as possible, the TensorFlow team has created the tf_upgrade_v2 utility to help transition legacy code to the new API.
Keras is a high-level neural network API, written in Python and capable of running on top of TensorFlow. Please check the software modules list via
marie@compute$ module spider Keras [...]
to find out, which Keras modules are available on your partition. TensorFlow should be automatically loaded as a dependency. After loading the module, you can use Keras as usual.