As soon as you have access to ZIH systems, you have to manage your data. Several filesystems are available. Each filesystem serves for special purpose according to their respective capacity, performance and permanence.
||4 PB||global||No||Only accessible via Workspaces. Not made for billions of files!|
||40 TB||global||No||Only accessible via Workspaces. For small I/O operations|
||232 TB||global||No||Only accessible via Workspaces. Fastest available filesystem, only for large parallel applications running with millions of small I/O operations|
||95 GB||local||No||is cleaned up after the job automatically|
Recommendations for Filesystem Usage¶
To work as efficient as possible, consider the following points
- Save source code etc. in
- Store checkpoints and other temporary data in
- Compilation in
Getting high I/O-bandwidth
- Use many clients
- Use many processes (writing in the same file at the same time is possible)
- Use large I/O transfer blocks
Cheat Sheet for Debugging Filesystem Issues¶
Users can select from the following commands to get some idea about their data.
For the first view, you can use the command
Alternatively, you can use the command
findmnt, which is also able to report space usage
by adding the parameter
marie@login$ findmnt -D
Optionally, you can use the parameter
-t to specify the filesystem type or the parameter
alter the output.
Do not use the
du-command for this purpose. It is able to cause issues
for other users, while reading data from the filesystem.