Invited Speakers
Antonio Gonzalez
Universitat Politècnica de Catalunya
Title: Removing Ineffectual Computations in
Neural Networks
Abstract
There is a growing interest in extending computing devices with
the ability to analyze and understand signals and data coming from
a large variety of activities in our daily live, and provide real
time responses in complex situations, with the goal to emulate
human perception and problem solving. Examples include personal
assistants, self-driving cars, domestics robots and health-care
devices just to name a few. Neural networks have proven to be an
effective approach to support many of these functionalities.
Most of these systems have very limited energy budgets so the
effectiveness of this approach is strongly dependent on the
energy-efficiency of the adopted solution. In this talk we present
several alternative directions for improving the energy-efficiency
of neural networks based on identifying and removing ineffectual
computations.
Bio
Antonio González (Ph.D. 1989) is a Full Professor at the Computer
Architecture Department of the Universitat Politècnica de
Catalunya, Barcelona (Spain), and the director of the Architecture
and Compiler research group. He was the founding director of the
Intel Barcelona Research Center from 2002 to 2014. His research
has focused on computer architecture. In this area, Antonio holds
52 patents, has published over 370 research papers and has given
over 120 invited talks. He has also made multiple contributions to
the design of the architecture of several commercial
microprocessors.
Antonio has been program chair for ICS, ISPASS, MICRO, HPCA and
ISCA, and general chair for MICRO and HPCA among other symposia.
He has served on the program committee for over 130 international
symposia in the field of computer architecture, and has been
Associate Editor of the IEEE Transactions on Computers, IEEE
Transactions on Parallel and Distributed Systems, IEEE Computer
Architecture Letters, ACM Transactions on Architecture and Code
Optimization, ACM Transactions on Parallel Computing, and Journal
of Embedded Computing.
Antonio’s awards include the award to the best student in computer
engineering in Spain, the Rosina Ribalta award as the advisor of
the best PhD project in Information Technology and Communications,
the Duran Farrell award for research in technology, the Aritmel
National Award of Informatics to the Computer Engineer of the
Year, the King James I award for his contributions in research on
new technologies, and the ICREA Academia Award. He is an IEEE
Fellow.
David Kaeli
Northeastern University
Title: Scaling Machine Learning Workloads on
Today’s GPUs
Abstract
Machine learning applications place large computational demands on
hardware resources when performing classification, regression,
clustering and training. What is common in many of these
applications is that the quality of the outcome or model improves
as we process more data. GPUs have been shown to be an effective
platform for accelerating machine learning workloads, though have
limits in terms of the amount a single GPU can process. This talk
will look at ongoing work in hardware compaction and multi-GPU
acceleration, enabling further scaling of machine learning
workloads.
Bio
David Kaeli received his BS and PhD in Electrical Engineering from
Rutgers University, and an MS in Computer Engineering from
Syracuse University. He is presently a COE Distinguished Full
Processor on the ECE faculty at Northeastern University, Boston,
MA. Dr. Kaeli has published over 350 critically reviewed
publications, 7 books, and 13 patents. He serves as the Editor in
Chief of ACM Transactions on Computer Architecture and Code
Optimization, and an Associate Editor of the IEEE Transactions on
Parallel and Distributed Systems and the Journal of Parallel and
Distributed Computing. Dr. Kaeli is an IEEE Fellow and an ACM
Distinguished Scientist.
Tushar Krishna
Georgia Tech
Title: A Communication-Centric Approach for
Designing Flexible DNN Accelerators
Abstract
Deep Neural Networks (DNN) have demonstrated highly promising
results across computer vision and speech recognition, and are
becoming foundational for ubiquitous AI. The computational
complexity of these algorithms and a need for high
energy-efficiency has led to a surge in research on hardware
accelerators. To reduce the latency and energy costs of accessing
DRAM, most DNN accelerators are spatial in nature, with hundreds
of processing elements (PE) operating in parallel and
communicating with each other directly.
DNNs are evolving at a rapid rate - leading to myriad layer types
(convolution, attention, LSTM, MLP) of varying shape (regular and
irregular). Given a DNN there can be myriad computationally
efficient implementations (e.g., via pruning) - leading to
structured and unstructured sparsity. Finally, a given DNN can be
tiled and partitioned in myriad ways to exploit data reuse. All of
the above can lead to irregular dataflow patterns within the
accelerator substrate. Getting high mapping efficiency for all
these cases is highly challenging in accelerators today that are
often tightly coupled 2D grids with rigid near-neighbor
connectivity.
First, given a target DNN, we will demonstrate a systematic
methodology for understanding data reuse opportunities within the
algorithm and determine the cost vs benefit for efficiently
exploiting them in hardware using our dataflow +
microarchitectural model called MAESTRO (MICRO 2019 + IEEE Micro
Top Picks). Next, we present a systematic communication-centric
methodology for accelerator design, that can provide ~100%
efficiency for arbitratry DNNs shapes, sparsity ratios and
mappings. We demonstrate instances of this approach with two
accelerators, MAERI (ASPLOS 2018 + IEEE Micro Top Picks Hon’
mention) and SIGMA (HPCA 2020 + Best Paper Award), that show
orders of magnitude better utilization over state-of-the-art
baselines like NVIDIA's NVDLA and Google’s TPU.
Bio
Tushar Krishna is an Assistant Professor in the School of
Electrical and Computer Engineering at Georgia Tech. He also holds
the ON Semiconductor Junior Professorship. He has a Ph.D. in
Electrical Engineering and Computer Science from MIT (2014), a
M.S.E in Electrical Engineering from Princeton University (2009),
and a B.Tech in Electrical Engineering from the Indian Institute
of Technology (IIT) Delhi (2007). Before joining Georgia Tech in
2015, Dr. Krishna spent a year as a post-doctoral researcher at
Intel, Massachusetts.
Dr. Krishna’s research spans computer architecture,
interconnection networks, networks-on-chip (NoC) and deep learning
accelerators with a focus on optimizing data movement in modern
computing systems. Three of his papers have been selected for IEEE
Micro’s Top Picks from Computer Architecture, one more received an
honorable mention, and three have won best paper awards. He
received the National Science Foundation (NSF) CRII award in 2018,
and both a Google Faculty Award and a Facebook Faculty Award in
2019.
Cliff Young
Title: Reflections on TPUs, Current Problems
in Acceleration, and What's Next
Abstract
Google's first TPU has been a remarkably successful accelerator,
spawning a sequence of successors and inspiring a wave of new
chips from established companies and startups. I'll start with
some retrospection about what we got right and the ways in which
we were lucky in building that first TPU. Then I'll pivot to the
problems I think are currently hard and possibly underserved by
our NN accelerator systems (to spoil: programmability, memory, and
multi-tenancy). Lastly I'll speculate about where ML might take
us: how much might the algorithms and computations change, the
implications of the Accelerator Wall, and the virtuous feedback
between algorithms and architecture that might be the basis of a
true Golden Age for our field.
Bio
Cliff Young is a software engineer in the Google Brain team, where
he works on codesign for deep learning accelerators. He is one of
the designers of Google’s Tensor Processing Unit (TPU), which is
used in production applications including Search, Maps, Photos,
and Translate. TPUs also powered AlphaGo’s historic 4-1 victory
over Go champion Lee Sedol. Previously, Cliff built
special-purpose supercomputers for molecular dynamics at D. E.
Shaw Research and worked at Bell Labs. Cliff holds AB, MS, and PhD
degrees in computer science from Harvard University.