Foreword

This compendium is work in progress, since we try to incorporate more information with increasing experience and with every question you ask us. We invite you to take part in the improvement of these pages by correcting or adding useful information or commenting the pages.

Ulf Markwardt

Contents

• Introduction
• Access, login, project management, step-by step examples
• Our HPC Systems
  • Taurus: general purpose HPC cluster (HRSK-II)
  • Venus: SGI Ultraviolet
• Hardware
• DataManagement

• BatchSystems
• RuntimeEnvironment
• SoftwareDevelopment
  • BuildingSoftware
  • GPUProgramming
• Checkpoint/Restart
• Containers

• Available Software
• FurtherDocumentation

News

• 2017-09-14: Introduction to HPC - slides
• 2017-08-08:
  New Hardware: We now have 32 nodes with Intel Broadwell CPUs (28 cores and 64 GB RAM, see https://doc.zih.tu-dresden.de/hpc-wiki/bin/view/Compendium/HardwareTaurus#broadwell). To explicitely run a job here, use the Slurm option "-p broadwell". Without a partition flag, Slurm automatically schedules jobs to these nodes according to the memory requirements. This integration and a Slurm update last week have been responsible for a few minor interruptions in the batch system... thank you for your patience!
  New Module Environment: We have decided to make LMOD the new default for our environment modules infrastructure on Taurus. Its main advantage is a higher speed at commands like "module avail" or the tab completion due to the caching of module files. (https://doc.zih.tu-dresden.de/hpc-wiki/bin/view/Compendium/RuntimeEnvironment#Lmod:_An_Alternative_Module_Implementation)
  Checkpoint/Restart: We are aiming to reduce the maximum job run-time in our cluster step by step. In order to help you shorten your individual jobs, we now offer a solution to automatically create chain jobs with generic checkpoint/restart out of normal batch scripts. There are still some limitations with regard to MPI-applications, but we'd be very interested in your feedback for your sequential/multi-threaded applications. You can find detailed documentation in our HPC wiki. (https://doc.zih.tu-dresden.de/hpc-wiki/bin/view/Compendium/CheckpointRestart)
  Containers: We now have established Singularity as container environment on Taurus. Your existing Docker containers can easily be converted. This can also be very useful to make software run on Taurus that has several distribution specific dependencies which are not easily installable in our RHEL-based environment, as well as for circumenventing problems of binary-distributed software that, e.g., depends on a newer GLIBC version than what is currently available on Taurus. (https://doc.zih.tu-dresden.de/hpc-wiki/bin/view/Compendium/Container)