MARGE – Multi-Access Research Gateway for HPC Experts

Link to the service

URL (currently out of order): http://138.246.238.140/ 

Documentation: https://gitlab.lrz.de/nfdi4ing/marge

Logo
Detailed description of the service

Data from High-Performance Computing (HPC) and High-Performance Measurement is often stored in personal (project) accounts at the computing centers and is often immobile due to its size. Access to the data or further use was previously not possible without an explicit request to the computing centers. Tier 0 systems in general can only be accessed after submitting a project application.

In high-performance computing, dealing with vast amounts of data, often in the range of several terabytes, is not uncommon, posing a challenge as it exceeds the storage capacity of typical consumer electronic hard drives. Furthermore, sharing such substantial datasets becomes a formidable task. To address this issue, we have devised a cloud-based solution leveraging the LRZ Compute Cloud to facilitate access to data stored in the LRZ DSS. 

MARGE working principles

We’ve developed a tool specifically designed for LRZ users, enabling them to grant external users, who may not have an LRZ account, access to their data through the compute cloud. External users obtain access to the cloud and establish a connection to specific data stored at the LRZ. In this arrangement, external users have read-only access to the data, allowing them to execute or develop their own analysis tools within the cloud. They can assess the shared data, perform analyses, and subsequently download the results they obtain. This advancement significantly enhances the ability to share extensive datasets among research colleagues.

The cloud server system enables research data generated at the Leibniz Supercomputing Centers (LRZ) to be made available to external parties. Access can be set up individually via a virtual machine or microservice. Thanks to the direct integration into the data storage infrastructure of the computing centers, both “hot” and “cold” HPC research data can be accessed. It is also possible to analyze the data on the virtual machine.

MARGE ParaView Visualizer

The cloud system offers the following usage options:

  • Provision of exclusive usage rights within a professionally managed compute cloud.

  • Execution of virtual machines on the cloud for on-site data analysis

  • Fast connection to existing LRZ data storage systems

MARGE large data
Terms of use & restrictions
References

publications that reference (or report on using) the service

  1. Reference on Service: Christof Bless, Ildar Baimuratov, and Oliver Karras: SciKGTeX – A LaTeX Package to Semantically Annotate Contributions in Scientific Publications.In: 2023 ACM/IEEE Joint Conference on Digital Libraries (JCDL), IEEE, 2023, https://dx.doi.org/10.1109/JCDL57899.2023.00030

  2. Reference on Use: Christof Bless, I. Baimuratov, and O. Karras: SciKGTeX – Scientific Knowledge Graph TeX, Computer software, https://github.com/Christof93/SciKGTeX

  3. Reference on Use: O. Karras, A. Ferrari, D. Fucci, and D. Dell’Anna: Supplementary Materials of the Tutorial: “Promotion of Open Science in Requirements Engineering – Leveraging the Open Research Knowledge Graph for FAIR Scientific Information” (1.1), 32nd IEEE International Requirements Engineering Conference 2024 (RE’24), Reykjavik, Iceland. Zenodo, 2023. https://doi.org/10.5281/zenodo.12518069

#WhyNFDI

Bring the user to the data

Miscellaneous

 

Tags

NFDI4ING services may be relevant to different users according to varying requirements. To support filtering or sorting, we added a tag system outlining which archetype, phase of the data lifecycle, or degree of maturity a service corresponds to. By clicking on one of the tags below, you can get an overview of all services aligned with each tag.

This service has the following tags:

The tags correspond to:
The Archetypes: Services relevant to Alex – Bespoke Experiments, Betty – Research Software Engineering, Caden – Provenance Tracking, Doris – High Performance Computing, Ellen – Complex Systems, Fiona – Data Re-Use and Enrichment

The data lifecycle: Services related to Informing & Planning, Organising & Processing, Describing & Documenting, Storing & Computing,
Finding & Re-Using, Learning & Teaching

The maturity of the service: Services sorted according to their maturity and status of their integration into the larger NFDI service landscape. For this we use the Integration Readiness Level (IRL), ranging from IRL0 (no specifications, strictly internal use) up to IRL4 (fully integrated in the German research data landscape and the EOSC). Click here for a diagram outlining all Integration Readiness Levels.