Machine learning research with positive real-world impact.

We develop industry leading machine learning systems. Our initiatives have the power to uplift large populations, while advancing the field of artificial intelligence.

Ambitious applied research, positive outcomes

Recent highlights

Our research is supported by access to massive datasets, close collaboration with world renowned academic faculty, and a uniquely scalable machine learning platform.

ICLR 2023 Spotlight | Disparate Impact in Differential Privacy from Gradient Misalignment

ICLR 2023 Spotlight | Disparate Impact in Differential Privacy from Gradient Misalignment

ICLR 2023 Spotlight | Disparate Impact in Differential Privacy from Gradient Misalignment

Abstract

As machine learning becomes more widespread throughout society, aspects including data privacy and fairness must be carefully considered, and are crucial for deployment in highly regulated industries. Unfortunately, the application of privacy enhancing technologies can worsen unfair tendencies in models. In particular, one of the most widely used techniques for private model training, differentially private stochastic gradient descent (DPSGD), frequently intensifies disparate impact on groups within data. In this work we study the fine-grained causes of unfairness in DPSGD and identify gradient misalignment due to inequitable gradient clipping as the most significant source. This observation leads us to a new method for reducing unfairness by preventing gradient misalignment in DPSGD.

ICLR 2023 | Verifying the Union of Manifolds Hypothesis for Image Data

ICLR 2023 | Verifying the Union of Manifolds Hypothesis for Image Data

ICLR 2023 | Verifying the Union of Manifolds Hypothesis for Image Data

Abstract

Deep learning has had tremendous success at learning low-dimensional representations of high-dimensional data. This success would be impossible if there was no hidden low-dimensional structure in data of interest; this existence is posited by the manifold hypothesis, which states that the data lies on an unknown manifold of low intrinsic dimension. In this paper, we argue that this hypothesis does not properly capture the low-dimensional structure typically present in image data. Assuming that data lies on a single manifold implies intrinsic dimension is identical across the entire data space, and does not allow for subregions of this space to have a different number of factors of variation. To address this deficiency, we consider the union of manifolds hypothesis, which states that data lies on a disjoint union of manifolds of varying intrinsic dimensions. We empirically verify this hypothesis on commonly-used image datasets, finding that indeed, observed data lies on a disconnected set and that intrinsic dimension is not constant. We also provide insights into the implications of the union of manifolds hypothesis in deep learning, both supervised and unsupervised, showing that designing models with an inductive bias for this structure improves performance across classification and generative modelling tasks. Our code is available at https://github.com/layer6ai-labs/UoMH.

ICLR 2023 | Temporal Dependencies in Feature Importance for Time Series Prediction

ICLR 2023 | Temporal Dependencies in Feature Importance for Time Series Prediction

ICLR 2023 | Temporal Dependencies in Feature Importance for Time Series Prediction

Abstract

Time series data introduces two key challenges for explainability methods: firstly, observations of the same feature over subsequent time steps are not independent, and secondly, the same feature can have varying importance to model predictions over time. In this paper, we propose Windowed Feature Importance in Time (WinIT), a feature removal based explainability approach to address these issues. Unlike existing feature removal explanation methods, WinIT explicitly accounts for the temporal dependence between different observations of the same feature in the construction of its importance score. Furthermore, WinIT captures the varying importance of a feature over time, by summarizing its importance over a window of past time steps. We conduct an extensive empirical study on synthetic and real-world data, compare against a wide range of leading explainability methods, and explore the impact of various evaluation strategies. Our results show that WinIT achieves significant gains over existing methods, with more consistent performance across different evaluation metrics.

ICML 2022 | Bayesian Nonparametrics for Offline Skill Discovery

ICML 2022 | Bayesian Nonparametrics for Offline Skill Discovery

ICML 2022 | Bayesian Nonparametrics for Offline Skill Discovery

Abstract

Skills or low-level policies in reinforcement learning are temporally extended actions that can speed up learning and enable complex behaviours. Recent work in offline reinforcement learning and imitation learning has proposed several techniques for skill discovery from a set of expert trajectories. While these methods are promising, the number K of skills to discover is always a fixed hyperparameter, which requires either prior knowledge about the environment or an additional parameter search to tune it. We first propose a method for offline learning of options (a particular skill framework) exploiting advances in variational inference and continuous relaxations. We then highlight an unexplored connection between Bayesian nonparametrics and offline skill discovery, and show how to obtain a nonparametric version of our model. This version is tractable thanks to a carefully structured approximate posterior with a dynamically-changing number of options, removing the need to specify K. We also show how our nonparametric extension can be applied in other skill frameworks, and empirically demonstrate that our method can outperform state-of-the-art offline skill learning algorithms across a variety of environments.

TMLR 2022 | Diagnosing and Fixing Manifold Overfitting in Deep Generative Models

TMLR 2022 | Diagnosing and Fixing Manifold Overfitting in Deep Generative Models

TMLR 2022 | Diagnosing and Fixing Manifold Overfitting in Deep Generative Models

Abstract

Likelihood-based, or explicit, deep generative models use neural networks to construct flexible high-dimensional densities. This formulation directly contradicts the manifold hypothesis, which states that observed data lies on a low-dimensional manifold embedded in high-dimensional ambient space. In this paper we investigate the pathologies of maximum-likelihood training in the presence of this dimensionality mismatch. We formally prove that degenerate optima are achieved wherein the manifold itself is learned but not the distribution on it, a phenomenon we call manifold overfitting. We propose a class of two-step procedures consisting of a dimensionality reduction step followed by maximum-likelihood density estimation, and prove that they recover the data-generating distribution in the nonparametric regime, thus avoiding manifold overfitting. We also show that these procedures enable density estimation on the manifolds learned by implicit models, such as generative adversarial networks, hence addressing a major shortcoming of these models. Several recently proposed methods are instances of our two-step procedures; we thus unify, extend, and theoretically justify a large class of models.

Our research areas include recommendation systems, computer vision, time series forecasting, and natural language processing.

View all publications

Big vision, deep roots

The co-founders of Layer 6, Jordan Jacobs and Tomi Poutanen, are also founders of the Vector Institute for Artificial Intelligence, and we maintain multiple research initiatives with Vector faculty. Current and former scientific advisors include professors Raquel Urtasun, Sanja Fidler, Rich Zemel, David Duvenaud, Laura Rosella and Scott Sanner.

Meaningful partnerships

Originally founded in 2011, Layer 6 now forms the AI research lab of TD Bank Group. Layer 6 impacts the lives of 25 million customers, helping more people achieve their financial goals. Partnerships with TD Securities provides Layer 6 with market data for training algo trading systems.

Layer 6 embraces opportunities to collaborate with Toronto’s world-leading medical research community, offering deep learning solutions to transform healthcare delivery and improve health outcomes. We are the first to deploy deep learning models on health data covering large population.

Passion to learn, driven to succeed

Our team represents 18 different countries of birth and we care deeply about fostering an inclusive culture where we learn from each other and win together. We are united by our passion for deep learning and a desire to apply our skills to have an outsized and positive impact on the future.

Meet some of our team

  • Tomi Poutanen

    Co-founder, Layer 6
    Chief AI Officer, TD Bank Group

    Tomi is a co-founder of Layer 6 which was acquired by TD Bank Group in January 2018 where he serves as the Chief AI Officer. Prior to Layer 6 Tomi founded three technology companies with exits to Microsoft and Yahoo!. Tomi is a co-founder of the Vector Institute, a world leading academic research institute for deep learning. He is a founding fellow at the Creative Destruction Lab, the world’s largest AI venture accelerator. Tomi holds an MSc in Computer Engineering and an MBA from University of Toronto. Most importantly, Tomi is a husband and a father to two daughters.

    LinkedIn Github Google Research

  • Maksims Volkovs

    Co-founder & Head of Machine Learning, Layer 6
    Chief AI Scientist, TD Bank Group

    Maks is a co-founder of Layer 6 and the Chief AI Scientist at TD Bank Group. Maks received his PhD from the University of Toronto where he was part of the Machine Learning Group led by Geoffrey Hinton. He previously worked at Microsoft Research and at Credit Suisse in London, UK. Maks is the winner or medalist in multiple international machine learning competitions run by Kaggle (where his global peak ranking was #47 of over 75,000 data scientists), ACM RecSys (including winner of the 2017 and 2018 RecSys Challenge) and others. He has published over 15 papers and is co-inventor of 6 patents.

    LinkedIn Github Google Research Website

  • Laura Doktorova

    Chief Architect

    Laura is the Chief Architect at Layer 6. She has a solid track record of building efficient technologies. Prior to Layer 6, Laura worked at Milq Inc. as the Chief Architect for 4 years and at Blackberry in various engineering roles for 10 years. Laura is passionate about entrepreneurship. She is a co-founder of Bebedo whose Inquiro™ technology provides high-speed spatio-temporal search on large volumes of time and location sensitive data. Laura received her degree from Belarusian State University of Informatics and Radioelectronics.

    LinkedIn Github

  • Harry Braviner

    Machine Learning Scientist

    Harry is a Machine Learning Scientist at Layer 6. He is a mathematician and astrophysicist by training, and received his PhD, MMath and BA from Trinity College at the University of Cambridge. Before moving to Canada, Harry lived in London and worked for the research arm of a systematic hedge fund. His current research is focussed on reinforcement learning, and teaching machines to solve difficult control problems.

    LinkedIn Github

  • Guangwei Yu

    Machine Learning Scientist

    Guang is leading the Computer Vision team at Layer 6. The Team achieved 2nd place at the Google Landmark Retrieval Challenge and 4th place at the RSNA Pneumonia Detection Challenge in 2018. Guang is also the medalist in multiple international machine learning competitions including the only repeat winner of the ACM RecSys Challenge and others. Guang received his MScAC in Computer Science and BASc in Engineering Science from University of Toronto.

    LinkedIn Github Google Research Website

  • Saba Zuberi

    Machine Learning Scientist

    Saba is a Machine Learning Scientist at Layer 6. She has years of data science experience and has previously worked at TD Bank Group in Toronto and at TaskRabbit in Silicon Valley. Saba is a Physicist with a PhD degree from University of Toronto. She completed her Post Doc research at the Lawrence Berkeley National Laboratory in California.

    LinkedIn Github

  • Yichao Lu

    Machine Learning Scientist

    Yichao joined Layer 6 as a Machine Learning Scientist after a research internship at Layer 6 where he collaborated closely with the research fellows at the Vector Institute. Yichao received his MScAC degree in Computer Science at the University of Toronto. Before moving to Canada, Yichao lived in Shanghai and completed his bachelor’s degree from Fudan University.

    LinkedIn Github

Develop your career at Layer 6

What’s in a name? Deep learning is inspired by the human brain. The cortex has six layers of neurons, where the sixth layer integrates the deeper processing to produce an action.

We’re growing our team exclusively with people driven to be at the top of the game in machine learning.

View current openings

In the news

Obtenez des renseignements en français

Layer 6 s’est engagé à servir la communauté francophone au meilleur de nos capacités. Pour obtenir des renseignements en français sur notre équipe ou des opportunités de carrière, veuillez écrire à l’adresse info@layer6.ai.