Alan Turing Institute Deep Learning Workshop, Edinburgh 2015

Summary description

Deep Learning is the learning and use of highly flexible multilayer models for machine learning. Deep learning has proven itself, both in terms of academic research and capability, and in terms of commercial interest. The Deep Learning Scoping Workshop will work on establishing areas of joint research and furthering UK capacity and capability building in this field. It will link established and junior researchers, build links with commercial research groups and introduce members to new problems and data sources. The aim of the workshop is to ensure UK researchers are on the crest of the wave of deep learning. We will plan for an ATI theme in deep learning, agree a process on early and regular communication, establish common research themes, and propose potential funding bids.

Key scientific questions to be answered

The workshop will focus on four aspects of deep learning. First, we will consider the further generalisations of deep learning methods beyond classification and feature generation, to areas of generative modelling, transfer learning, multimodal integration, reinforcement learning and other important problems. Second, we will ask what other practical problems/scenarios can benefit from the application of deep learning methods beyond current domains, and how models can be distilled and made accessible for use in those domains. Third, we will propose potential approaches for reducing or distributing the computational costs associated with deep learning. Fourth we will examine the theoretical aspects of deep learning, and ask why deep learning has proven so effective and what the limitations are.

Key additional topics to be addressed

In addition to addressing the key scientific questions above, we will discuss the potential benefits of coordinated activity. The topics of this discussion will include

Facilitating trusted early communication of approaches and results to enable the UK to maintain its position in the fast and competitive research in this field.
The potential to maintain joint data repositories that will provide benefits to all parties.
The steep learning curve for doctoral students and postdocs. The working environment is quickly changing. What coordination will helps ensure researchers are quickly up to speed?
What interaction is required with business research labs and startups to enable them to take up deep learning methods, where otherwise they would not have the opportunity.
What the best practice is, in training the next generation in deep learning.

Workshop

The workshop follows up on the 1st Edinburgh Deep Learning Workshop in 2014, and the 2nd Edinburgh Deep Learning Workshop earlier this year.

The workshop will be in the School of Informatics, University of Edinburgh. Those involved with contributing to organising the workshop include Amos Storkey (Edinburgh) and Krzysztof Geras (Edinburgh), who are the local organisers, and Nando De Freitas (Oxford, DeepMind), Ben Graham (Warwick), Zoubin Ghahramani (Cambridge), Thore Graepel (UCL, DeepMind), Neil Lawrence (Sheffield), Phil Blunsom (Oxford, DeepMind), Iain Murray (Edinburgh), Stephen Roberts (Oxford), Andrew Zisserman (Oxford, DeepMind), Mark Gales (Cambridge), Vittorio Ferrari (Edinburgh), Yee Whye Teh (Oxford), Andrea Vedaldi (Oxford), Steve Renals (Edinburgh), Andrew Stuart (Warwick), Charles Sutton (Edinburgh), Richard Turner (Cambridge), Chris Williams (Edinburgh), Emre Ozer (ARM), Max Welling (Amsterdam).

Preliminary programme:

9:15-9:30 - introductory talk by Amos Storkey,
9:30-10:15 - Yee Whye Teh (University of Oxford), "Distributed Bayesian Deep Learning via a convergent Monte Carlo/Expectation Propagation hybrid",
10:15-10:45 - Alison Lowndes (NVIDIA), "Accelerating deep learning",
10:45-11:15 - coffe break,
11:15-11:45 - Yarin Gal (University of Cambridge), "What My Deep Model Doesn't Know...",
11:45-12:00 - Harri Edwards (University of Edinburgh), "Censoring Representations with an Adversary",
12:00-12:15 - Anders Boesen Lindbo Larsen (DTU), "Dreaming More Data: Class-dependent Distributions over Diffeomorphisms for Learned Data Augmentation",
12:15-12:30 - Tim Rocktäschel (UCL), "Reasoning about Entailment with Neural Attention",
12:30-13:30 - lunch,
13:30-14:30 - poster session,
14:30-15:00 - Ben Graham (University of Warwick), "Spatially Sparse Convolutional Neural networks",
15:00-15:30 - Taco Cohen (University of Amsterdam), "Group-Equivariant Convolutional Networks",
15:30-16:30 - panel discussion.

Posters:

Tree Recurrent Neural Networks with Application to Language Modeling
Xingxing Zhang, Liang Lu, Mirella Lapata
Strategic Dialogue Management via Deep Reinforcement Learning
Heriberto Cuayahuitl, Simon Keizer, Oliver Lemon
Dreaming More Data: Class-dependent Distributions over Diffeomorphisms for Learned Data Augmentation
Soren Hauberg, Oren Freifeld, Anders Boesen Lindbo Larsen, John W. Fisher III, Lars Kai Hansen
Predicting Embedded Syntactic Structures from Natural Language Sentences with Neural Network Approaches
Gregory Senay, Fabio Massimo Zanzotto, Lorenzo Ferrone, Luca Rigazio
Backpropagating at the edge of chaos
Victor Hernandez-Urbina
Reasoning about Entailment with Neural Attention
Tim Rocktaschel, Edward Grefenstette, Karl Moritz Hermann,Tomas Kocisky, Phil Blunsom
Should We Learn Dropout?
Gavin Gray, Amos Storkey
Compressing LSTMs into CNNs
Krzysztof J. Geras, Abdel-rahman Mohamed, Rich Caruana, Gregor Urban, Shengjie Wang, Ozlem Aslan, Matthai Philipose, Matthew Richardson, Charles Sutton

Costs

The workshop registration is free of charge to registered attendees.

Travel

Participants will need to make their own travel arrangements. By train the nearest station is Edinburgh Waverley, which is less than a 15 minute walk from the forum. See National Rail Enquiries for train information. For those coming from further afield, information about travel to and from Edinburgh Airport is available. A taxi to the city centre from the airport costs about 22GBP to 24GBP one way. There is an express bus from the airport called Airlink that terminates in the city centre and costs 7GBP for a return journey. The journey to the airport requires approximately 30 minutes from the city centre of actual journey time (add a little more during the rush hours). There is also a tram that goes to the centre of town. It takes a little longer than the bus, and is slightly more expensive.

Submission

If you wish to contribute a poster to this workshop, then please send a title and one or two page description (or a paper in conference paper format) to amos+deep@@inf.ed.ac.uk, replacing the dual @ sign. We will then be in contact about your submission. It is likely that not all submissions will be able to be included.

Registration

If you wish to attend the open session on the 25th November then please register at the eventbrite site. Those who have been invited directly should not use this link.

Please also join our Facebook group

Alan Turing Institute Deep Learning Open Workshop

Edinburgh, 25 November 2015