Frequently Asked Questions
Account registration
Connecting
How can I log into the machines at the MPCDF?
What are the gateway machines for SSH login?
Two-factor authentication (2FA)
Are SSH keys supported?
Can ssh/scp/sftp performance be improved?
I get a SSH security warning about a host key change when trying to log in. What does this mean and what should I do?
How can I run applications with graphical user interfaces on MPCDF systems?
X11 forwarding
VNC
Remote Visualization Service
Two-factor authentication (2FA)
General information about 2FA
Do I need to enable 2FA?
Do I already have 2FA activated?
Why is 2FA enforced?
What is a token, OTP, and seed?
What kinds of tokens are there?
Why am I asked for an OTP?
Activation of 2FA
How do I enable 2FA?
How do I enroll and use an app token?
What app do I need to install?
How do I register my existing token?
How do I enroll and use a secondary/backup token?
Why can’t I have both an app token and a hardware token?
2FA Tips and Tricks
Do I have to type in an OTP every time I access the secured systems?
2FA Troubleshooting
I need to factory-reset my phone. How can I preserve my app token?
How do I transfer my token to a new phone?
I can’t validate/activate the token I just scanned (“Wrong OTP”)
I can’t log in to the SelfService anymore
I can’t log in to a gate machine via SSH
I can’t access the HPC clusters through VNC anymore
Hardware and client support
How can I use my sshfs/rsync/scp/sftp GUI with 2FA?
How can I use FileZilla with 2FA?
How to use 2FA on a phone with a time shift?
Do you or are you planning to support FIDO2/U2F/YubiKeys?
On security
If the seed is stored on the server, how secure is it?
Where do the hardware tokens come from? Does the vendor know the seeds?
HPC Software and Applications
General questions
How and where can I install software myself?
Environment modules
How can I use environment modules interactively?
How can I use environment modules in scripts?
How can I use environment modules to avoid absolute paths in my scripts?
Examples
How do the hierarchical environment modules work?
How do I quickly find a module?
How can I disable the “MPCDF specific note” for
module
avail
?
Why there are no BLAS/LAPACK modules available on the clusters?
Compiled Languages
CMake
C/C++ and Fortran
Debugging of C/C++ and Fortran codes
Interpreted Languages
Python
R
Julia
Jupyter Notebooks
MATLAB
Message Passing Interface (MPI) Libraries and Runtimes
Which MPI implementations are supported?
How can I compile and link my MPI application?
CMake cannot find MPI, what can I do?
I cannot launch my MPI code using
mpirun
, what am I doing wrong?
Visualization
How can I combine many image files (JPG, PNG) into a movie (MP4)?
How can I install additional packages for TeX/LaTeX locally?
GUI applications
Visual Studio Code (in combination with the Remote-SSH extension) does not work any more on certain HPC clusters
Why do certain GUI applications not work (any more) on the login nodes of HPC clusters? What are the alternatives?
HPC Systems and Services
Raven
What are the recommended compiler optimization flags for Raven?
Slurm batch system
How do I submit jobs to the Slurm batch system?
Can I submit jobs longer than 24 hours? Why are job run times limited to 24 hours?
How do I launch an MPI code?
What is the correct order of executable commands and ‘#SBATCH’ directives in a Slurm job script?
Can I work interactively for debugging and development work?
How can I query the estimated start time of my job?
How can I get information about my job at runtime?
How can I get information about my job after it has finished?
What happens when a running job is hit by a hardware failure?
How do I do pinning correctly?
Parallel File Systems (GPFS)
Which file systems are available and how should I use them?
I do not get the IO performance I would expect. Am I doing something wrong?
How can I grant other users access to my files? How do I use ACLs?
How can I transfer files to and from the HPC systems?
Performance Monitoring
How can I find out about the performance of my runs?
How can I stop the background performance monitoring?
GPU Computing
How can I launch the NVIDIA Multi-Process Service (MPS)?
How can I profile my GPU code?
Are there dedicated GPU resources available for interactive GPU development?
Containers
Which container solutions are supported? Can I run Docker containers?
Remote Visualization
How can I run visualization tools or any software that uses OpenGL on MPCDF systems?
How can I get access to MPCDF remote visualization services?
Hints, Tips, and Tricks
How can I change my default shell?
How can I launch
ssh-agent
in a terminal session?
How can I avoid having to type my password repeatedly? How can I tunnel through the gateway machines?
How can I access the clusters using a Windows machine?
Help, Support, and Training
How can I get Help and Support?
Are you offering trainings on how to use MPCDF resources?
How to acknowledge MPCDF?
Documentation
Computing
Introduction
Overview
Compute Facilities
Application Support for HPC, AI and HPDA
Gateway machines
Login
GSSAPI-based logins to MPCDF hosts
Tunneled access to MPCDF services
Viper User Guide
System overview
Access
Hardware configuration
File systems
Software
Slurm batch system
Slurm example batch scripts
Migration guide for users coming from Intel-based HPC systems
Raven User Guide
System Overview
Access
Hardware configuration
File systems
Software
Slurm batch system
Slurm example batch scripts
Dedicated clusters
Astronomy
Astrophysics
Biochemistry
Biological Cybernetics
Biological Intelligence
Biophysics
Brain Research
Chemical Physics of Solids
Extraterrestrial Physics
FHI / MPIE / MPIP
Gravitational Physics
Gravitational Physics - ACR
Gravitational Physics - CRA
Iron Research
MPSD / PKS
Neurobiology of Behavior – caesar
Physics
Plasma Physics
Psychiatry
Quantum Optics
Radioastronomy
Science of Light
Software
Environment Modules
HPC Application Packages
Compilers and languages
Libraries
Debugging tools
Performance tools
Mathematical tools
Bioinformatics
Containers
Data Analytics / Machine Learning
VNC
Quickstart guide to HPC
Software environment (modules)
MPI parallel HPC applications
Multithreaded (OpenMP) or hybrid (MPI/OpenMP) HPC applications
Execution of (parallel) programs via Slurm
Performance Monitoring
Introduction
PDF Performance Reports for Users
Suspending the Performance Monitoring System for Specific Jobs
Technical Background
Overhead
Further information
Training
Courses and workshops arranged by or in collaboration with the MPCDF
Training programmes of other institutions
Data
DataShare: Sync and Share Service
DataShare: An Introduction
Update the domain of the client
FAQ
Desktop and Mobile clients
Globus Online: Large-Scale data Transfer and Sharing
MPCDF DataHub and Globus Online
Staging Files to HPC systems via Globus Online
Staging data via Flows
GO-Nexus
Globus Demo Videos - Demonstrating Globus Functionality for end users
Nexus-S3: Object Storage for data Transfer and Sharing
Nexus-S3
Publishing Data for public access via S3
Small to Medium Scale Data Transfers
Data Transfer: Tools & Tips
MPCDF DataHub and Globus Online
Sharing Large Files with DataShare
GitLab: Software Development
The MPCDF GitLab Instance: an introduction
Poetry and GitLab: Devops for Python developers
GitLab Runners for CI/CD
Data Publication and Metadata Management
Service: Data Repositories
The MPCDF Metadata Tools: User Documentation
The MPCDF Metadata Tools: Developer Documentation
MetaStore User Documentation
Backup and Archive
Backup & Archive Systems
How to archive data
Backup HPC
Backup Linux Clusters
Backup AFS
Backup Desktops
Deprecated: The Andrew File System (AFS)
Store (AFS)
Introduction
Specific-Technical
Glossary
Troubleshooting
HPC-Cloud
Technical and User Documentation
Quick Start
Compute
Storage
Network
Command Line Interface and Scripting
Recipes
Rental Model
Introduction
Cost components
Setup procedure and billing procedure
Available Resources
Compute resources
Storage resources
Terms of Use
General
Backup and Recovery
Data Privacy and Sensitive Data
Proprietary Software
Service Interventions and Scheduled Down times
Performance
Support
Potential Sanctions
Visualization
Support for the Visualization of Scientific Data
Remote Visualization and Jupyter Notebook Services
Web Interface
Command line interface
Technical details
Troubleshooting
Robin
Campus
Software
VPN
VPN Server (Gateway)
Connection Profile (Group)
Desktop and Notebook (Windows, Mac OS X, Linux)
Linux Clients
Mobile Devices (Android and iOS)
Wi-Fi
Guest networks
Eduroam
Installation (
MPCDF/MPQ staff only
)
Email Configuration
Email Service
Email Client configuration
Mailing Lists
Bits and Bytes
No.216, August 2024
High-performance Computing
New HPC system
Viper
(Phase-1: CPU)
HPC Software News
MPCDF GitLab
module-image
to be discontinued on October 31
New module images available in MPCDF GitLab
Nvidia HPC SDK version 24.3 available on
Raven
Major Change in the Python Infrastructure on the HPC Clusters
Introducing containerized Applications as Part of the MPCDF Software Stack
Nexus-S3: Object Storage in the HPC-Cloud and beyond
Example use case
Opt-in via SelfService
Object storage for larger projects
The MetaStore Research Data Publication Platform
Managing data sets and resources in MetaStore
Who can use it
News
Password policy
2FA for DataShare and GitLab
Events
AMD GPU workshop & hackathon (November 5-7)
Introduction to MPCDF services (October 24)
Meet MPCDF
MPCDF at Garching Campus Open Doors (October 3)
HPC-Cloud workshop (September 10-12)
Course on “Python for HPC” (November 26-28)
No.215, April 2024
High-performance computing
Licensed software in Slurm (Comsol)
HPC Software News
Improved workflow for multimer predictions with AlphaFold2 on
Raven
New version of Intel oneAPI with deprecation of ifort compiler
CUDA modules on
Raven
New AMD-GPU ELPA release
Kubernetes in the HPC-Cloud
Usage
Function
Existing applications
GitLab: Graphs & Diagrams
Alternatives
LLMs meet MPCDF
News & Events
Introduction to MPCDF services
Meet MPCDF
No.214, December 2023
HPC Software News
CUDA-aware OpenMPI on
Raven
GPU-accelerated VASP
Intel oneAPI: transition from ifort to ifx
Module Software Stacks for Continuous Integration Pipelines on MPCDF GitLab Shared Cloud Runners
Introducing a novel module-enabled Docker image infrastructure
Announcing legacy status and later discontinuation of the
module-image
Compressed Portable Conda Environments for HPC Systems
Introduction and Motivation
Move Conda environments into compressed image files
Basic usage examples
Limitations
Availability
New Features in the HPC-Cloud
Expanded menu of flavors and images
SSD-based block volumes
Automated domain name service
Shared filesystem service
The Robin cluster
News & Events
AMD-GPU development workshop
Meet MPCDF
RDA Deutschland Tagung 2024
No.213, August 2023
High-performance Computing
New GPU development partition on
Raven
Memory profiling with heaptrack
New compilers and libraries: Intel oneAPI 2023.1
Using linters to improve and maintain code quality
HPC-Cloud Object Storage
JADE - Automated Slurm deployments in the HPC-Cloud
GitLab: Tips & Tricks
Online editing of source code revisited
Custom badges
Security warning
New IBM tape library and tape drives installed at MPCDF
News & Events
Open positions at MPCDF
Meet MPCDF
AMD-GPU development workshop
No.212, April 2023
High-performance Computing
New Supercomputer of the MPG -
Cobra
successor
Documentation of HPC hardware characteristics
CMake Recipes Repository
MPCDF HPC Cloud
Introduction
Hardware Resources
Project Support
Summary
News
MPCDF SelfService
Pushing Fusion-Plasma Simulations Towards Exascale
Base4NFDI: Creating NFDI-wide basic services in a world of specific domains
Events
Meet MPCDF
Introduction to MPCDF services
AI Training Course
Course on “Python for HPC”
RDA-Deutschland Conference
No.211, December 2022
High-performance Computing
Anaconda Python modules
Hints for architecture-specific and optimized CUDA compilation
New Intel C/C++ compilers and associated MPCDF software stack
Turbomole license for MPG renewed
Apptainer on HPC clusters, the Linux Foundation successor to Singularity
GitLab Tips & Tricks: Use of Docker Images in GitLab CI
GO-Nexus
News & Events
Discontinuation of General VPN
Meet MPCDF
Python for HPC
Advanced HPC Workshop
No.210, August 2022
High-performance Computing
Cobra successor procurement
CO
2
footprint of MPCDF
Software news
GitLab CI Distributed Cache
Introduction
Adding a cache to your CI configuration
CI of software that depends on third-party packages
CI of a complex HPC code that requires CPU and partly GPU resources
Concluding remarks
Globus Flows
Introduction
Flows in detail
Example
Summary
News & Events
Introduction to MPCDF services
Meet MPCDF
Advanced HPC workshop
Python for HPC
No.209, April 2022
High-performance Computing
AlphaFold2 on the HPC system
Raven
GitLab CI
GitLab shared runners on GPUs
Continuous integration testing for HPC codes on MPCDF GitLab
Globus Online
DataHub access via the Globus Online Portal
Enhanced functionality
More information
New SelfService Features and Improvements
Redesign of the login process
Viewing accounting data
Additional improvements
Access to AFS restricted for local Access only
News & Events
AI bootcamp
International HPC Summer School 2022
Workshop “Introduction to MPCDF services (online)”
“Meet MPCDF”: New online forum and lectures for MPCDF users
RDA-Deutschland-Tagung 2022
No.208, December 2021
High-performance Computing
Termination of general user operation for
Draco
login nodes
Announcement of CUDA no-defaults on
Cobra
and
Raven
Usage of /tmp and /dev/shm on
Cobra
and
Raven
Eigensolver library ELPA
Using Python-based hybrid-parallel codes on HPC systems
Python bindings for C++ using pybind11 and scikit-build
Interfacing Python/NumPy with C++ using pybind11
Build with scikit-build
The Gitlab Package Registry
Example: Publishing Python packages
Using Application Tokens instead of Passwords
DataShare
GitLab
Software Publishing
Do it yourself
Publish via a data archiving site
Software Heritage
Final remarks
News & Events
International HPC Summer School 2022
Advanced HPC workshop 2021
60 Years Max Planck Computing Centre in Garching
No.207, August 2021
High-performance Computing
HPC system
Raven
fully operational
GPU Computing on
Raven
HPC system
Cobra
- Module system to be aligned with
Raven
Decommissioning of
Draco
HPC Cloud
Poetry: Packaging and Dependency Management for Python
Introduction
A Poetry project and initial configuration
Dependency Management
Running your code
Package creation and publishing
Conclusion
More functionality for the SelfService - Call to action for 2FA users
Migration of MyMPCDF functionality
Updated password policy
Two-Factor Authentication (2FA) - Call to action
News & Events
Brochure “High-Performance Computing and Data Science in the MPG”
GPU bootcamp
Introductory course for new users of MPCDF
Advanced HPC workshop
Python for HPC
No.206, April 2021
High-performance Computing
HPC System
Raven
: deployment of the final system
Charliecloud and Singularity containers supported on
Cobra
and
Raven
Control and verification of the CPU affinity of processes and threads
High-performance data analytics and AI software stack at MPCDF
Decommissioning of AFS
Relaunch of MPCDF website and new technical documentation platform
Events
New online introductory course for new users of MPCDF
Advanced HPC workshop: save the date
Previous Editions
Technical Documentation
Documentation
Computing
Dedicated clusters
Astrophysics
Astrophysics
Name of the clusters:
Astrophysics FREYA
Astrophysics ORION
Astrophysics VIRGO
Institution:
Max Planck Institute for Astrophysics