BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Europe/Stockholm
X-LIC-LOCATION:Europe/Stockholm
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20191206T142452Z
LOCATION:Analog 1\, 2
DTSTART;TZID=Europe/Stockholm:20180625T135300
DTEND;TZID=Europe/Stockholm:20180625T135700
UID:isc_hpc_ISC High Performance 2018_sess103_phd107@linklings.com
SUMMARY:(PhD14) Hybrid Hierarchical Data Management System: Accelerating D
 ata Processing on HPC Systems
DESCRIPTION:PhD Forum\n\n(PhD14) Hybrid Hierarchical Data Management Syste
 m: Accelerating Data Processing on HPC Systems\n\nCheng, Lu, Du\n\nHigh-pe
 rformance computing (HPC) systems provide an indispensable computing infra
 structure for scientific modeling and simulations. While HPC systems are i
 nvented to solve compute-intensive workloads, the explosion of scientific 
 data generated from large-scale simulations and advanced sensors makes sci
 entific workflows more complex and more data-intensive.\nSystem architectu
 res are designed to best support the typical workloads running on the clus
 ters. Supporting these coupled workflows that contain both compute-intensi
 ve and data-intensive workloads on HPC systems presents new challenges in 
 data management due to their scales, coordination behaviors and overall co
 mplexities. \n1) Firstly, the increasing amounts of data are likely to sto
 re in different data sources, including cloud storage, databases and diver
 se file systems. The lack of uniform data access interface leads to furthe
 r increase in workflow complexity. \n2) Secondly, the performance gap betw
 een memory and disk keeps growing, it’s urgent to provide an efficient dat
 a sharing mechanism that allows different workload to share data in memory
  without repetitive data movement. \n3) Thirdly, heterogeneous storage dev
 ices (e.g., memory, SSDs, HDDs) are commonly used and the memory storage h
 ierarchy on HPC systems is getting deeper, it’s critical for data manageme
 nt systems that can manage data on tired storage efficiently.\n4)Finally, 
 existing HPC parallel file systems, such as Lustre, GPFS, and OrangeFS, fa
 ce serious challenges in managing metadata because of the centralized meta
 data management.\nScientific data management can be divided into several s
 tages, including data collection, storing &querying, data processing, publ
 ish and sharing, etc. A few data management systems have explored some of 
 these aspects. Storage Resource Broker, developed by San Diego Supercomput
 er Center, is a middleware that provides uniform API that allows users to 
 access heterogeneous storage resources. iRODS, the successor of SRB, provi
 de a configurable rule engine to set customized data management rules. Whi
 le SRB and iRODS focus on data storing and data sharing, Qserv and FastQue
 ry focus on data querying to satisfy the ad-hoc user query access to gigan
 tic scientific data. These data management systems simplified data sharing
 , but provide limited support to accelerate data processing.\nTo solve the
  above challenges and accelerate data processing of coupled workflows on H
 PC systems, we intend to design Hybrid Hierarchical Data Management System
 , which contains following modules:\n	Tiered Data Management provi
 des cross-tier data sharing on available storage devices to reduce data mo
 vement and customizes data management strategies based on workflow data ac
 cess patterns to accelerate data processing.\n	Data-aware Job Sche
 duling aims to move the compute to the data and make job scheduling decisi
 ons based on storage tier and workflow data access patterns.\n	Dis
 tributed Metadata Management aims to solve the centralized metadata manage
 ment bottleneck via Hash+SubTree Namespace management. All metadata are st
 ored in Key-Value pair to satisfied the diverse metadata requirement on sc
 ientific data.\n	Domain Specific Optimizations Module provides spe
 cific optimizations for target applications, such as optimizations for Had
 oop/Spark shuffle, optimizations for lots of small files.\n\nPasses: Confe
 rence Pass, Exhibition Pass, Big Data Analytics, HPC workflows, Storage Te
 chnologies, System Software & Runtime Systems\n\nTag: Conference Pass, Exh
 ibition Pass, Big Data Analytics, HPC workflows, Storage Technologies, Sys
 tem Software & Runtime Systems
URL:https://2018.isc-program.com/presentation/?id=phd107&sess=sess103
END:VEVENT
END:VCALENDAR