Machine learning research with positive real-world impact.
Ambitious applied research, positive outcomes
Recent highlights
Our research is supported by access to massive datasets, close collaboration with world renowned academic faculty, and a uniquely scalable machine learning platform.
ICLR 2023 Spotlight | Disparate Impact in Differential Privacy from Gradient Misalignment
ICLR 2023 Spotlight | Disparate Impact in Differential Privacy from Gradient Misalignment

Abstract
As machine learning becomes more widespread throughout society, aspects including data privacy and fairness must be carefully considered, and are crucial for deployment in highly regulated industries. Unfortunately, the application of privacy enhancing technologies can worsen unfair tendencies in models. In particular, one of the most widely used techniques for private model training, differentially private stochastic gradient descent (DPSGD), frequently intensifies disparate impact on groups within data. In this work we study the fine-grained causes of unfairness in DPSGD and identify gradient misalignment due to inequitable gradient clipping as the most significant source. This observation leads us to a new method for reducing unfairness by preventing gradient misalignment in DPSGD.
ICLR 2023 | Verifying the Union of Manifolds Hypothesis for Image Data
ICLR 2023 | Verifying the Union of Manifolds Hypothesis for Image Data

Abstract
Deep learning has had tremendous success at learning low-dimensional representations of high-dimensional data. This success would be impossible if there was no hidden low-dimensional structure in data of interest; this existence is posited by the manifold hypothesis, which states that the data lies on an unknown manifold of low intrinsic dimension. In this paper, we argue that this hypothesis does not properly capture the low-dimensional structure typically present in image data. Assuming that data lies on a single manifold implies intrinsic dimension is identical across the entire data space, and does not allow for subregions of this space to have a different number of factors of variation. To address this deficiency, we consider the union of manifolds hypothesis, which states that data lies on a disjoint union of manifolds of varying intrinsic dimensions. We empirically verify this hypothesis on commonly-used image datasets, finding that indeed, observed data lies on a disconnected set and that intrinsic dimension is not constant. We also provide insights into the implications of the union of manifolds hypothesis in deep learning, both supervised and unsupervised, showing that designing models with an inductive bias for this structure improves performance across classification and generative modelling tasks. Our code is available at https://github.com/layer6ai-labs/UoMH.
ICLR 2023 | Temporal Dependencies in Feature Importance for Time Series Prediction
ICLR 2023 | Temporal Dependencies in Feature Importance for Time Series Prediction

Abstract
Time series data introduces two key challenges for explainability methods: firstly, observations of the same feature over subsequent time steps are not independent, and secondly, the same feature can have varying importance to model predictions over time. In this paper, we propose Windowed Feature Importance in Time (WinIT), a feature removal based explainability approach to address these issues. Unlike existing feature removal explanation methods, WinIT explicitly accounts for the temporal dependence between different observations of the same feature in the construction of its importance score. Furthermore, WinIT captures the varying importance of a feature over time, by summarizing its importance over a window of past time steps. We conduct an extensive empirical study on synthetic and real-world data, compare against a wide range of leading explainability methods, and explore the impact of various evaluation strategies. Our results show that WinIT achieves significant gains over existing methods, with more consistent performance across different evaluation metrics.
ICML 2022 | Bayesian Nonparametrics for Offline Skill Discovery
ICML 2022 | Bayesian Nonparametrics for Offline Skill Discovery

Abstract
Skills or low-level policies in reinforcement learning are temporally extended actions that can speed up learning and enable complex behaviours. Recent work in offline reinforcement learning and imitation learning has proposed several techniques for skill discovery from a set of expert trajectories. While these methods are promising, the number K of skills to discover is always a fixed hyperparameter, which requires either prior knowledge about the environment or an additional parameter search to tune it. We first propose a method for offline learning of options (a particular skill framework) exploiting advances in variational inference and continuous relaxations. We then highlight an unexplored connection between Bayesian nonparametrics and offline skill discovery, and show how to obtain a nonparametric version of our model. This version is tractable thanks to a carefully structured approximate posterior with a dynamically-changing number of options, removing the need to specify K. We also show how our nonparametric extension can be applied in other skill frameworks, and empirically demonstrate that our method can outperform state-of-the-art offline skill learning algorithms across a variety of environments.
TMLR 2022 | Diagnosing and Fixing Manifold Overfitting in Deep Generative Models
TMLR 2022 | Diagnosing and Fixing Manifold Overfitting in Deep Generative Models

Abstract
Likelihood-based, or explicit, deep generative models use neural networks to construct flexible high-dimensional densities. This formulation directly contradicts the manifold hypothesis, which states that observed data lies on a low-dimensional manifold embedded in high-dimensional ambient space. In this paper we investigate the pathologies of maximum-likelihood training in the presence of this dimensionality mismatch. We formally prove that degenerate optima are achieved wherein the manifold itself is learned but not the distribution on it, a phenomenon we call manifold overfitting. We propose a class of two-step procedures consisting of a dimensionality reduction step followed by maximum-likelihood density estimation, and prove that they recover the data-generating distribution in the nonparametric regime, thus avoiding manifold overfitting. We also show that these procedures enable density estimation on the manifolds learned by implicit models, such as generative adversarial networks, hence addressing a major shortcoming of these models. Several recently proposed methods are instances of our two-step procedures; we thus unify, extend, and theoretically justify a large class of models.
Our research areas include recommendation systems, computer vision, time series forecasting, and natural language processing.

Big vision, deep roots
The co-founders of Layer 6, Jordan Jacobs and Tomi Poutanen, are also founders of the Vector Institute for Artificial Intelligence, and we maintain multiple research initiatives with Vector faculty. Current and former scientific advisors include professors Raquel Urtasun, Sanja Fidler, Rich Zemel, David Duvenaud, Laura Rosella and Scott Sanner.
Meaningful partnerships
Originally founded in 2011, Layer 6 now forms the AI research lab of TD Bank Group. Layer 6 impacts the lives of 25 million customers, helping more people achieve their financial goals. Partnerships with TD Securities provides Layer 6 with market data for training algo trading systems.
Layer 6 embraces opportunities to collaborate with Toronto’s world-leading medical research community, offering deep learning solutions to transform healthcare delivery and improve health outcomes. We are the first to deploy deep learning models on health data covering large population.
Passion to learn, driven to succeed
Our team represents 18 different countries of birth and we care deeply about fostering an inclusive culture where we learn from each other and win together. We are united by our passion for deep learning and a desire to apply our skills to have an outsized and positive impact on the future.
Meet some of our team
Develop your career at Layer 6
We’re growing our team exclusively with people driven to be at the top of the game in machine learning.

In the news
-
Inside TD’s AI play: How Layer 6’s technology hopes to improve old-fashioned banking advice
Globe And Mail
Read articleon Globe And Mail -
Tomi Poutanen: Geoffrey Hinton's Turing Award celebrates a life devoted to ground-breaking AI research
TD Newsroom
Read articleon TD Newsroom -
Tomi Poutanen: Chief Artificial Intelligence Officers Enter the C-Suite
Wall Street Journal
Read articleon Wall Street Journal -
TD Bank’s ‘Layer 6’ to bring machine learning personalization to diabetes care
IT Business
Read articleon IT Business -
TD Advances Innovation in Canadian Healthcare
TD Bank Group
Read articleon TD Bank Group
-
Inside TD’s AI play: How Layer 6’s technology hopes to improve old-fashioned banking advice
Globe And Mail
Read articleon Globe And Mail -
Tomi Poutanen: Geoffrey Hinton's Turing Award celebrates a life devoted to ground-breaking AI research
TD Newsroom
Read articleon TD Newsroom -
Tomi Poutanen: Chief Artificial Intelligence Officers Enter the C-Suite
Wall Street Journal
Read articleon Wall Street Journal -
TD Bank’s ‘Layer 6’ to bring machine learning personalization to diabetes care
IT Business
Read articleon IT Business -
TD Advances Innovation in Canadian Healthcare
TD Bank Group
Read articleon TD Bank Group -
Jordan Jacobs, co-founder of Vector Institute on Canada as a global AI leader
IT Business
Read articleon IT Business -
Layer 6’s Jordan Jacobs: Canada needs to promote itself as an AI leader
BetaKit
Read articleon BetaKit -
U of T alumni and graduate students part of Layer 6 AI's win in global competition
U of T News
Read articleon U of T News -
Tomi Poutanen and Michael Rhodes discuss the future of artificial intelligence with Amanda Lang
TD Bank Group
Read articleon TD Bank Group -
Get Smart – Artificial intelligence is transforming business and life
Ivey Business School
Read articleon Ivey Business School -
How TD Bank plans to use artificial intelligence
BNN Bloomberg
Read articleon BNN Bloomberg