BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Europe/Stockholm
X-LIC-LOCATION:Europe/Stockholm
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20191206T142453Z
LOCATION:Substanz 1\, 2
DTSTART;TZID=Europe/Stockholm:20180626T083000
DTEND;TZID=Europe/Stockholm:20180626T100000
UID:isc_hpc_ISC High Performance 2018_sess113_post127@linklings.com
SUMMARY:(RP01) Can Unified-Memory support on Pascal and Volta GPUs enable 
 Out-of-Core DNN Training?
DESCRIPTION:Research Poster\n\n(RP01) Can Unified-Memory support on Pascal
  and Volta GPUs enable Out-of-Core DNN Training?\n\nAwan, Chu, Subramoni, 
 Lu, Panda\n\nExisting Deep Neural Network (DNN) training frameworks like C
 affe and CNTK\ncannot train large DNNs that do not fit the GPU memory with
 out explicit memory\nmanagement schemes. In this poster, we propose OC-DNN
  - a novel\nOut-of-Core DNN training framework that\nexploits the new Unif
 ied Memory (UM) features (since CUDA 8) along with new\nhardware mechanism
 s in Pascal and Volta GPUs. OC-DNN has two major\ndesign components --- 1)
  OC-Caffe; an enhanced version of Caffe that\nexploits innovative UM featu
 res like asynchronous prefetching, managed page\nmigration, and exploitati
 on of GPU-based page faults and cudaMemAdvise interface\nto enable efficie
 nt out-of-core training for very large DNNs and/or DNNs that\nrequire larg
 e batch sizes, and 2) an interception library to transparently\nleverage t
 hese cutting-edge features for several other DNN frameworks without\nany d
 esign changes. To the best of our knowledge, this is the first attempt to\
 ndesign an out-of-core DNN training framework that exploits CUDA UM interf
 ace in\ntandem with page migration and prefetching capabilities of Pascal/
 Volta GPUs to\ndeal with memory-bound out-of-core DNNs with high-performan
 ce and\nhigh-productivity. We provide a comprehensive performance characte
 rization of\nour designs. OC-Caffe provides comparable performance (to Caf
 fe) for regular\nDNNs. OC-Caffe-Opt is up to 1.9X faster than OC-Caffe-Nai
 ve and up\nto 5X faster than optimized CPU-based training for out-of-core 
 workloads.\nOC-Caffe also allows scale-up (DGX-1) and scale-out on multi-G
 PU clusters.\n\nPasses: Conference Pass, AI/Machine Learning/Deep Learning
 , Computer Architecture, HPC Accelerators, MPI, Programming Models & Langu
 ages\n\nTag: Conference Pass, AI/Machine Learning/Deep Learning, Computer 
 Architecture, HPC Accelerators, MPI, Programming Models & Languages
URL:https://2018.isc-program.com/presentation/?id=post127&sess=sess113
END:VEVENT
END:VCALENDAR