Heterogeneous systems combining GPUs and CPUs are becoming mainstream. The performance potential of these processors working together is significant, yet most applications choose to use such systems as homogeneous, exploiting either GPUs or CPUs exclusively. This choice is due to the perceived imbalance between the high difficulty of deploying heterogeneous applications and the unknown performance gain. The main goal of this tutorial is to demonstrate that implementing and deploying heterogeneous applications is simpler than expected, is supported by easy-to-use tools, and leads to significant performance gain for many applications. Therefore, our tutorial has three parts: (1) introduce the principles of heterogeneous computing models and sketch the current landscape, (2) demonstrate how to select the most suitable model for a given workload, and (3) present in more detail the representative models that can efficiently cover most applications. When discussing the principles and design of modern heterogeneous computing, we cover static workload partitioning, dynamic run-time based models, and hierarchical multi-node models. We further give representative examples from each class, focusing on their strengths and weaknesses. Further, we show how one can use semi-automated workload characterization to match applications with the right heterogeneous computing model. Finally, we demonstrate the usability and practical differences between tools by using hands-on exercises with several representative tools and up to 4 application case-studies, in single- and multi-node environments.

Tutorial organizers: Dr. ir. Ana Lucia Varbanescu, UvA, The Netherlands Dr. Jie Shen, NUDT, China

Ana Lucia Varbanescu is an assistant professor at University of Amsterdam, The Netherlands. She obtained her BSc and MSc from POLITEHNICA University of Bucharest, Romania. In 2010 she received her PhD from Delft University of Technology. She continued as a PostDoc researcher at VU University Amsterdam and Delft University of Technology, positions partially funded by a NWO IsFast grant. In 2012, she received a STW VENI grant. Since 2013, she is a MacGillavry fellow at University of Amsterdam. She is an active participant in the ASCI doctoral school, ICT.Open, and ICT with Industry. From 2007 till 2014, she was a visiting researcher at IBM T.J.Watson (USA), NVIDIA (USA), BSC (Spain), and Imperial College of London (UK). Her research interests include high performance computing, parallel programming, performance analysis and prediction, and big data analytics.