Alan Turing Institute Deep Learning Open Workshop

Edinburgh, 25 November 2015

Ground floor, Informatics Forum, 10 Crichton Street, Edinburgh
Directions to the Informatics Forum are available

Local Organisers: Amos Storkey, Krzysztof J. Geras

Summary description

Deep Learning is the learning and use of highly flexible multilayer models for machine learning. Deep learning has proven itself, both in terms of academic research and capability, and in terms of commercial interest. The Deep Learning Scoping Workshop will work on establishing areas of joint research and furthering UK capacity and capability building in this field. It will link established and junior researchers, build links with commercial research groups and introduce members to new problems and data sources. The aim of the workshop is to ensure UK researchers are on the crest of the wave of deep learning. We will plan for an ATI theme in deep learning, agree a process on early and regular communication, establish common research themes, and propose potential funding bids.

Key scientific questions to be answered

The workshop will focus on four aspects of deep learning. First, we will consider the further generalisations of deep learning methods beyond classification and feature generation, to areas of generative modelling, transfer learning, multimodal integration, reinforcement learning and other important problems. Second, we will ask what other practical problems/scenarios can benefit from the application of deep learning methods beyond current domains, and how models can be distilled and made accessible for use in those domains. Third, we will propose potential approaches for reducing or distributing the computational costs associated with deep learning. Fourth we will examine the theoretical aspects of deep learning, and ask why deep learning has proven so effective and what the limitations are.

Key additional topics to be addressed

In addition to addressing the key scientific questions above, we will discuss the potential benefits of coordinated activity. The topics of this discussion will include
  • Facilitating trusted early communication of approaches and results to enable the UK to maintain its position in the fast and competitive research in this field.
  • The potential to maintain joint data repositories that will provide benefits to all parties.
  • The steep learning curve for doctoral students and postdocs. The working environment is quickly changing. What coordination will helps ensure researchers are quickly up to speed?
  • What interaction is required with business research labs and startups to enable them to take up deep learning methods, where otherwise they would not have the opportunity.
  • What the best practice is, in training the next generation in deep learning.


The workshop follows up on the 1st Edinburgh Deep Learning Workshop in 2014, and the 2nd Edinburgh Deep Learning Workshop earlier this year.

The workshop will be in the School of Informatics, University of Edinburgh. Those involved with contributing to organising the workshop include Amos Storkey (Edinburgh) and Krzysztof Geras (Edinburgh), who are the local organisers, and Nando De Freitas (Oxford, DeepMind), Ben Graham (Warwick), Zoubin Ghahramani (Cambridge), Thore Graepel (UCL, DeepMind), Neil Lawrence (Sheffield), Phil Blunsom (Oxford, DeepMind), Iain Murray (Edinburgh), Stephen Roberts (Oxford), Andrew Zisserman (Oxford, DeepMind), Mark Gales (Cambridge), Vittorio Ferrari (Edinburgh), Yee Whye Teh (Oxford), Andrea Vedaldi (Oxford), Steve Renals (Edinburgh), Andrew Stuart (Warwick), Charles Sutton (Edinburgh), Richard Turner (Cambridge), Chris Williams (Edinburgh), Emre Ozer (ARM), Max Welling (Amsterdam).

Preliminary programme:
  • 9:15-9:30 - introductory talk by Amos Storkey,
  • 9:30-10:15 - Yee Whye Teh (University of Oxford), "Distributed Bayesian Deep Learning via a convergent Monte Carlo/Expectation Propagation hybrid",
  • 10:15-10:45 - Alison Lowndes (NVIDIA), "Accelerating deep learning",
  • 10:45-11:15 - coffe break,
  • 11:15-11:45 - Yarin Gal (University of Cambridge), "What My Deep Model Doesn't Know...",
  • 11:45-12:00 - Harri Edwards (University of Edinburgh), "Censoring Representations with an Adversary",
  • 12:00-12:15 - Anders Boesen Lindbo Larsen (DTU), "Dreaming More Data: Class-dependent Distributions over Diffeomorphisms for Learned Data Augmentation",
  • 12:15-12:30 - Tim Rocktäschel (UCL), "Reasoning about Entailment with Neural Attention",
  • 12:30-13:30 - lunch,
  • 13:30-14:30 - poster session,
  • 14:30-15:00 - Ben Graham (University of Warwick), "Spatially Sparse Convolutional Neural networks",
  • 15:00-15:30 - Taco Cohen (University of Amsterdam), "Group-Equivariant Convolutional Networks",
  • 15:30-16:30 - panel discussion.
  • Tree Recurrent Neural Networks with Application to Language Modeling
    Xingxing Zhang, Liang Lu, Mirella Lapata
  • Strategic Dialogue Management via Deep Reinforcement Learning
    Heriberto Cuayahuitl, Simon Keizer, Oliver Lemon
  • Dreaming More Data: Class-dependent Distributions over Diffeomorphisms for Learned Data Augmentation
    Soren Hauberg, Oren Freifeld, Anders Boesen Lindbo Larsen, John W. Fisher III, Lars Kai Hansen
  • Predicting Embedded Syntactic Structures from Natural Language Sentences with Neural Network Approaches
    Gregory Senay, Fabio Massimo Zanzotto, Lorenzo Ferrone, Luca Rigazio
  • Backpropagating at the edge of chaos
    Victor Hernandez-Urbina
  • Reasoning about Entailment with Neural Attention
    Tim Rocktaschel, Edward Grefenstette, Karl Moritz Hermann,Tomas Kocisky, Phil Blunsom
  • Should We Learn Dropout?
    Gavin Gray, Amos Storkey
  • Compressing LSTMs into CNNs
    Krzysztof J. Geras, Abdel-rahman Mohamed, Rich Caruana, Gregor Urban, Shengjie Wang, Ozlem Aslan, Matthai Philipose, Matthew Richardson, Charles Sutton


The workshop registration is free of charge to registered attendees.


Participants will need to make their own travel arrangements. By train the nearest station is Edinburgh Waverley, which is less than a 15 minute walk from the forum. See National Rail Enquiries for train information. For those coming from further afield, information about travel to and from Edinburgh Airport is available. A taxi to the city centre from the airport costs about 22GBP to 24GBP one way. There is an express bus from the airport called Airlink that terminates in the city centre and costs 7GBP for a return journey. The journey to the airport requires approximately 30 minutes from the city centre of actual journey time (add a little more during the rush hours). There is also a tram that goes to the centre of town. It takes a little longer than the bus, and is slightly more expensive.


If you wish to contribute a poster to this workshop, then please send a title and one or two page description (or a paper in conference paper format) to, replacing the dual @ sign. We will then be in contact about your submission. It is likely that not all submissions will be able to be included.


If you wish to attend the open session on the 25th November then please register at the eventbrite site. Those who have been invited directly should not use this link.

Please also join our Facebook group