Machine learning research with positive real-world impact.

We develop industry leading machine learning systems. Our initiatives have the power to uplift large populations, while advancing the field of artificial intelligence.

Ambitious applied research, positive outcomes

Recent highlights

Our research is supported by access to massive datasets, close collaboration with world renowned academic faculty, and a uniquely scalable machine learning platform.

CVPR 2024 Highlight  | Data-Efficient Multimodal Fusion on a Single GPU

CVPR 2024 Highlight  | Data-Efficient Multimodal Fusion on a Single GPU

CVPR 2024 Highlight  | Data-Efficient Multimodal Fusion on a Single GPU

Abstract

The goal of multimodal alignment is to learn a single latent space that is shared between multimodal inputs. The most powerful models in this space have been trained using massive datasets of paired inputs and large-scale computational resources, making them prohibitively expensive to train in many practical scenarios. We surmise that existing unimodal encoders pre-trained on large amounts of unimodal data should provide an effective bootstrap to create multimodal models from unimodal ones at much lower costs. We therefore propose FuseMix, a multimodal augmentation scheme that operates on the latent spaces of arbitrary pre-trained unimodal encoders. Using FuseMix for multimodal alignment, we achieve competitive performance — and in certain cases outperform state-of-the art methods — in both image-text and audio-text retrieval, with orders of magnitude less compute and data: for example, we outperform CLIP on the Flickr30K text-to-image retrieval task with ∼600× fewer GPU days and ∼80× fewer image-text pairs. Additionally, we show how our method can be applied to convert pre-trained text-to-image generative models into audio-to-image ones.

ICLR 2024 | Self-supervised Representation Learning from Random Data Projectors

ICLR 2024 | Self-supervised Representation Learning from Random Data Projectors

ICLR 2024 | Self-supervised Representation Learning from Random Data Projectors

Abstract

Self-supervised representation learning (SSRL) has advanced considerably by exploiting the transformation invariance assumption under artificially designed data augmentations. While augmentation-based SSRL algorithms push the boundaries of performance in computer vision and natural language processing, they are often not directly applicable to other data modalities, and can conflict with application-specific data augmentation constraints. This paper presents an SSRL approach that can be applied to any data modality and network architecture because it does not rely on augmentations or masking. Specifically, we show that high-quality data representations can be learned by reconstructing random data projections. We evaluate the proposed approach on a wide range of representation learning tasks that span diverse modalities and real-world applications. We show that it outperforms multiple state-of-the-art SSRL baselines. Due to its wide applicability and strong empirical results, we argue that learning from randomness is a fruitful research direction worthy of attention and further study.

TMLR 2024 | Neural Implicit Manifold Learning for Topology-Aware Density Estimation

TMLR 2024 | Neural Implicit Manifold Learning for Topology-Aware Density Estimation

TMLR 2024 | Neural Implicit Manifold Learning for Topology-Aware Density Estimation

Abstract

Natural data is often constrained to an low-dimensional manifold. This work focuses on the task of building theoretically principled generative models for such data. Current generative models learn the manifold by mapping an low-dimensional latent variable through a neural network. These procedures, which we call pushforward models, incur a straightforward limitation: manifolds cannot in general be represented with a single parameterization, meaning that attempts to do so will incur either computational instability or the inability to learn probability densities within the manifold. To remedy this problem, we propose to model the manifold as a neural implicit manifold: the set of zeros of a neural network. We then learn the probability density within the manifold with a constrained energy-based model, which employs a constrained variant of Langevin dynamics to train and sample from the learned manifold. In experiments on synthetic and natural data, we show that our model can learn manifold-supported distributions with complex topologies more accurately than pushforward models.

NeurIPS 2023 | Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models

NeurIPS 2023 | Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models

NeurIPS 2023 | Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models

Abstract

We systematically study a wide variety of image-based generative models spanning semantically-diverse datasets to understand and improve the feature extractors and metrics used to evaluate them. Using best practices in psychophysics, we measure human perception of image realism for generated samples by conducting the largest experiment evaluating generative models to date, and find that no existing metric strongly correlates with human evaluations. Comparing to 16 modern metrics for evaluating the overall performance, fidelity, diversity, and memorization of generative models, we find that the state-of-the-art perceptual realism of diffusion models as judged by humans is not reflected in commonly reported metrics such as FID. This discrepancy is not explained by diversity in generated samples, though one cause is over-reliance on Inception-V3. We address these flaws through a study of alternative self-supervised feature extractors, find that the semantic information encoded by individual networks strongly depends on their training procedure, and show that DINOv2-ViT-L/14 allows for much richer evaluation of generative models. Next, we investigate data memorization, and find that generative models do memorize training examples on simple, smaller datasets like CIFAR10, but not necessarily on more complex datasets like ImageNet. However, our experiments show that current metrics do not properly detect memorization; none in the literature is able to separate memorization from other phenomena such as underfitting or mode shrinkage. To facilitate further development of generative models and their evaluation we release all generated image datasets, human evaluation data, and a modular library to compute 16 common metrics for 8 different encoders.

NeurIPS 2023 | Adversarially robust learning with uncertain perturbation sets

NeurIPS 2023 | Adversarially robust learning with uncertain perturbation sets

NeurIPS 2023 | Adversarially robust learning with uncertain perturbation sets

Abstract

In many real-world settings exact perturbation sets to be used by an adversary are not plausibly available to a learner. While prior literature has studied both scenarios with completely known and completely unknown perturbation sets, we propose an in-between setting of learning with respect to a class of perturbation sets. We show that in this setting we can improve on previous results with completely unknown perturbation sets, while still addressing the concerns of not having perfect knowledge of these sets in real life. In particular, we give the first positive results for the learnability of infinite Littlestone classes when having access to a perfect-attack oracle. We also consider a setting of learning with abstention, where predictions are considered robustness violations, only when the wrong prediction is made within the perturbation set. We show there are classes for which perturbation-set unaware learning without query access is possible, but abstention is required.

Our research areas include recommendation systems, computer vision, time series forecasting, and natural language processing.

View all publications

Big vision, deep roots

The co-founders of Layer 6, Jordan Jacobs and Tomi Poutanen, are also founders of the Vector Institute for Artificial Intelligence, and we maintain multiple research initiatives with Vector faculty. Current and former scientific advisors include professors Raquel Urtasun, Sanja Fidler, Rich Zemel, David Duvenaud, Laura Rosella and Scott Sanner.

Meaningful partnerships

Originally founded in 2011, Layer 6 now forms the AI research lab of TD Bank Group. Layer 6 impacts the lives of 25 million customers, helping more people achieve their financial goals. Partnerships with TD Securities provides Layer 6 with market data for training algo trading systems.

Layer 6 embraces opportunities to collaborate with Toronto’s world-leading medical research community, offering deep learning solutions to transform healthcare delivery and improve health outcomes. We are the first to deploy deep learning models on health data covering large population.

Passion to learn, driven to succeed

Our team represents 18 different countries of birth and we care deeply about fostering an inclusive culture where we learn from each other and win together. We are united by our passion for deep learning and a desire to apply our skills to have an outsized and positive impact on the future.

Meet some of our team

  • Tomi Poutanen

    Co-founder, Layer 6
    Chief AI Officer, TD Bank Group

    Tomi is a co-founder of Layer 6 which was acquired by TD Bank Group in January 2018 where he serves as the Chief AI Officer. Prior to Layer 6 Tomi founded three technology companies with exits to Microsoft and Yahoo!. Tomi is a co-founder of the Vector Institute, a world leading academic research institute for deep learning. He is a founding fellow at the Creative Destruction Lab, the world’s largest AI venture accelerator. Tomi holds an MSc in Computer Engineering and an MBA from University of Toronto. Most importantly, Tomi is a husband and a father to two daughters.

    LinkedIn Github Google Research

  • Maksims Volkovs

    Co-founder & Head of Machine Learning, Layer 6
    Chief AI Scientist, TD Bank Group

    Maks is a co-founder of Layer 6 and the Chief AI Scientist at TD Bank Group. Maks received his PhD from the University of Toronto where he was part of the Machine Learning Group led by Geoffrey Hinton. He previously worked at Microsoft Research and at Credit Suisse in London, UK. Maks is the winner or medalist in multiple international machine learning competitions run by Kaggle (where his global peak ranking was #47 of over 75,000 data scientists), ACM RecSys (including winner of the 2017 and 2018 RecSys Challenge) and others. He has published over 15 papers and is co-inventor of 6 patents.

    LinkedIn Github Google Research Website

  • Laura Doktorova

    Chief Architect

    Laura is the Chief Architect at Layer 6. She has a solid track record of building efficient technologies. Prior to Layer 6, Laura worked at Milq Inc. as the Chief Architect for 4 years and at Blackberry in various engineering roles for 10 years. Laura is passionate about entrepreneurship. She is a co-founder of Bebedo whose Inquiro™ technology provides high-speed spatio-temporal search on large volumes of time and location sensitive data. Laura received her degree from Belarusian State University of Informatics and Radioelectronics.

    LinkedIn Github

  • Harry Braviner

    Machine Learning Scientist

    Harry is a Machine Learning Scientist at Layer 6. He is a mathematician and astrophysicist by training, and received his PhD, MMath and BA from Trinity College at the University of Cambridge. Before moving to Canada, Harry lived in London and worked for the research arm of a systematic hedge fund. His current research is focussed on reinforcement learning, and teaching machines to solve difficult control problems.

    LinkedIn Github

  • Guangwei Yu

    Machine Learning Scientist

    Guang is leading the Computer Vision team at Layer 6. The Team achieved 2nd place at the Google Landmark Retrieval Challenge and 4th place at the RSNA Pneumonia Detection Challenge in 2018. Guang is also the medalist in multiple international machine learning competitions including the only repeat winner of the ACM RecSys Challenge and others. Guang received his MScAC in Computer Science and BASc in Engineering Science from University of Toronto.

    LinkedIn Github Google Research Website

  • Saba Zuberi

    Machine Learning Scientist

    Saba is a Machine Learning Scientist at Layer 6. She has years of data science experience and has previously worked at TD Bank Group in Toronto and at TaskRabbit in Silicon Valley. Saba is a Physicist with a PhD degree from University of Toronto. She completed her Post Doc research at the Lawrence Berkeley National Laboratory in California.

    LinkedIn Github

  • Yichao Lu

    Machine Learning Scientist

    Yichao joined Layer 6 as a Machine Learning Scientist after a research internship at Layer 6 where he collaborated closely with the research fellows at the Vector Institute. Yichao received his MScAC degree in Computer Science at the University of Toronto. Before moving to Canada, Yichao lived in Shanghai and completed his bachelor’s degree from Fudan University.

    LinkedIn Github

Develop your career at Layer 6

What’s in a name? Deep learning is inspired by the human brain. The cortex has six layers of neurons, where the sixth layer integrates the deeper processing to produce an action.

We’re growing our team exclusively with people driven to be at the top of the game in machine learning.

View current openings

In the news

Obtenez des renseignements en français

Layer 6 s’est engagé à servir la communauté francophone au meilleur de nos capacités. Pour obtenir des renseignements en français sur notre équipe ou des opportunités de carrière, veuillez écrire à l’adresse info@layer6.ai.