In the first phase, the DEEP project laid the ground for the DEEP programming environment. Ater all, programming heterogeneous systems is often considered challenging and cumbersome. Therefore, the DEEP programming environment provides a dedicated development and runtime system which makes porting applications to DEEP project prototypes a snap. If your application supports MPI. If your application does not use MPI, the task-based OmpSs programming model might even save you from explicitly parallelizing your application yourself.


DEEP-ER: Resiliency & highly scalable I/O

In the second phase, within the DEEP-ER project, the system software stack was extended to feature a highly scalable, efficient, and user-friendly parallel I/O system based on the Fraunhofer parallel file system BeeGFS (formerly known as FhGFS). Additionally, it provides a low overhead, unified user-level checkpointing system and exploiting the multiple levels of non-volatile memory and storage added to the DEEP architecture.


DEEP-EST: Resource management & job scheduling

Now in the third phase, the DEEP-EST project, the focus in system software development lies with resource management and job scheduling. The developments in this area will enable to determine optimal resource allocations for each combination of workloads, support adaptive scheduling and enable dynamical reservation of the resources. Added to the modularity of the system itself, this guarantees maximum usage of the overall system, since no component is “blocked” for an application if it is not utilised.

Furthermore, the DEEP‑EST unified programming environment, based on standard components such as MPI and OmpSs but also on the emerging programming paradigms of the data-analytic fields like map-reduce, will provide a model that fully supports applications to use combinations of heterogeneous nodes, and enables developers to easily adapt and optimise their codes while keeping them fully portable.

On a Modular Supercomputer Architecture (MSA) system, applications will either use resources within a single module only, or run across different modules either at the same time, or successively in a workflow like model. This requires scalable scheduling and co-allocation of resources for jobs within and across modules.

Benchmarking is an essential element in evaluating the success of a hardware prototyping project. In the DEEP-ER project we use the JUBE benchmarking environment to assess the performance of the DEEP-ER system.

Exascale systems will require a combination of powerful resiliency techniques that are also flexible enough to accommodate the heterogeneous nature of such systems.

Read more ...

High frequency monitoring and operational data analytics in the DEEP projects is performed by the Data Center Data Base (DCDB) and its data anlytics framework Wintermute.

Highly-scalable, parallel I/O will be an important building block for future Exascale systems. In the DEEP-ER project, the I/O subsystem relies on 3 components: Fraunhofer's parallel file system BeeGFS, the parallel I/O library SIONlib, and Exascale10.

Read more ...