Machine learning research with positive real-world impact.

We develop industry leading machine learning systems. Our initiatives have the power to uplift large populations, while advancing the field of artificial intelligence.

Ambitious applied research, positive outcomes

Recent highlights

Our research is supported by access to massive datasets, close collaboration with world renowned academic faculty, and a uniquely scalable machine learning platform.

RecSys 2021 | User Engagement Modeling with Deep Learning and Language Models

RecSys 2021 | User Engagement Modeling with Deep Learning and Language Models

RecSys 2021 | User Engagement Modeling with Deep Learning and Language Models

Abstract

Twitter is one of the main information sharing platforms in the world with millions of tweets created daily. To ensure that users get relevant content in their feeds Twitter extensively leverages machine learning-based recommender systems. However, given the large volume of data, all production systems must be both memory and CPU efficient. In the 2021 ACM RecSys challenge Twitter simulates the production environment with a large dataset of almost 1 bilion user-tweet engagements that span a 4 week period. The goal is to accurately predict engagement type, and all models are subject to strict run-time constraints during inference. In this paper we present our approach to the 2021 ACM Recsys challenge. We use a hybrid pipeline and leverage gradient boosting, neural network classifiers and multi-lingual language models to maximize performance. Our approach achieves strong results placing 3’rd on the public leaderboard. We further explore the complexity of language model inference, and show that through distillation it can be possible to run such models in highly constrained production environments.

ICCV 2021 | Context-aware Scene Graph Generation with Seq2Seq Transformers

ICCV 2021 | Context-aware Scene Graph Generation with Seq2Seq Transformers

Context-aware Scene Graph Generation with Seq2Seq Transformers

Abstract

Scene graph generation is an important task in computer vision aimed at improving the semantic understanding of the visual world. In this task, the model needs to detect objects and predict visual relationships between them. Most of the existing models predict relationships in parallel assuming their independence. While there are different ways to capture these dependencies, we explore a conditional approach motivated by the sequence-to-sequence (Seq2Seq) formalism. Different from the previous research, our proposed model predicts visual relationships one at a time in an autoregressive manner by explicitly conditioning on the already predicted relationships. Drawing from translation models in NLP, we propose an encoder-decoder model built using Transformers where the encoder captures global context and long range interactions. The decoder then makes sequential predictions by conditioning on the scene graph constructed so far. In addition, we introduce a novel reinforcement learning-based training strategy tailored to Seq2Seq scene graph generation. By using a self-critical policy gradient training approach with Monte Carlo search we directly optimize for the (mean) recall metrics and bridge the gap between training and evaluation. Experimental results on two public benchmark datasets demonstrate that our Seq2Seq learning approach achieves strong empirical performance, outperforming previous state-of-the-art, while remaining efficient in terms of training and inference time.

NeurIPS 2021 | Rectangular Flows for Manifold Learning

NeurIPS 2021 | Rectangular Flows for Manifold Learning

NeurIPS 2021 | Rectangular Flows for Manifold Learning

Abstract

Normalizing flows are invertible neural networks with tractable change-of-volume terms, which allow optimization of their parameters to be efficiently performed via maximum likelihood. However, data of interest are typically assumed to live in some (often unknown) low-dimensional manifold embedded in a high-dimensional ambient space. The result is a modelling mismatch since — by construction — the invertibility requirement implies high-dimensional support of the learned distribution. Injective flows, mappings from low- to high-dimensional spaces, aim to fix this discrepancy by learning distributions on manifolds, but the resulting volume-change term becomes more challenging to evaluate. Current approaches either avoid computing this term entirely using various heuristics, or assume the manifold is known beforehand and therefore are not widely applicable. Instead, we propose two methods to tractably calculate the gradient of this term with respect to the parameters of the model, relying on careful use of automatic differentiation and techniques from numerical linear algebra. Both approaches perform end-to-end nonlinear manifold learning and density estimation for data projected onto this manifold. We study the trade-offs between our proposed methods, empirically verify that we outperform approaches ignoring the volume-change term by more accurately learning manifolds and the corresponding distributions on them, and show promising results on out-of-distribution detection. Our code is available at https://github.com/layer6ai-labs/rectangular-flows.

NeurIPS 2021 | Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows

NeurIPS 2021 | Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows

NeurIPS 2021 | Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows

Abstract

Normalizing flows are generative models that provide tractable density estimation via an invertible transformation from a simple base distribution to a complex target distribution. However, this technique cannot directly model data supported on an unknown low-dimensional manifold, a common occurrence in real-world domains such as image data. Recent attempts to remedy this limitation have introduced geometric complications that defeat a central benefit of normalizing flows: exact density estimation. We recover this benefit with Conformal Embedding Flows, a framework for designing flows that learn manifolds with tractable densities. We argue that composing a standard flow with a trainable conformal embedding is the most natural way to model manifold-supported data. To this end, we present a series of conformal building blocks and apply them in experiments with synthetic and real-world data to demonstrate that flows can model manifold-supported distributions without sacrificing tractable likelihoods.

CVPR 2021 | Weakly Supervised Action Selection Learning in Video

CVPR 2021 | Weakly Supervised Action Selection Learning in Video

Weakly Supervised Action Selection Learning in Video

Abstract

Localizing actions in video is a core task in computer vision. The weakly supervised temporal localization problem investigates whether this task can be adequately solved with only video-level labels, significantly reducing the amount of expensive and error-prone annotation that is required. A common approach is to train a frame-level classifier where frames with the highest class probability are selected to make a video-level prediction. Frame-level activations are then used for localization. However, the absence of frame-level annotations cause the classifier to impart class bias on every frame. To address this, we propose the Action Selection Learning (ASL) approach to capture the general concept of action, a property we refer to as “actionness”.  Under ASL, the model is trained with a novel class-agnostic task to predict which frames will be selected by the classifier. Empirically, we show that ASL outperforms leading baselines on two popular benchmarks THUMOS-14 and ActivityNet-1.2, with 10.3% and 5.7% relative improvement respectively. We further analyze the properties of ASL and demonstrate the importance of actionness.

Our research areas include recommendation systems, computer vision, time series forecasting, and natural language processing.

View all publications

Big vision, deep roots

The co-founders of Layer 6, Jordan Jacobs and Tomi Poutanen, are also founders of the Vector Institute for Artificial Intelligence, and we maintain multiple research initiatives with Vector faculty. Current and former scientific advisors include professors Raquel Urtasun, Sanja Fidler, Rich Zemel, David Duvenaud, Laura Rosella and Scott Sanner.

Meaningful partnerships

Originally founded in 2011, Layer 6 now forms the AI research lab of TD Bank Group. Layer 6 impacts the lives of 25 million customers, helping more people achieve their financial goals. Partnerships with TD Securities provides Layer 6 with market data for training algo trading systems.

Layer 6 embraces opportunities to collaborate with Toronto’s world-leading medical research community, offering deep learning solutions to transform healthcare delivery and improve health outcomes. We are the first to deploy deep learning models on health data covering large population.

Passion to learn, driven to succeed

Our team represents 18 different countries of birth and we care deeply about fostering an inclusive culture where we learn from each other and win together. We are united by our passion for deep learning and a desire to apply our skills to have an outsized and positive impact on the future.

Meet some of our team

  • Tomi Poutanen

    Co-founder, Layer 6
    Chief AI Officer, TD Bank Group

    Tomi is a co-founder of Layer 6 which was acquired by TD Bank Group in January 2018 where he serves as the Chief AI Officer. Prior to Layer 6 Tomi founded three technology companies with exits to Microsoft and Yahoo!. Tomi is a co-founder of the Vector Institute, a world leading academic research institute for deep learning. He is a founding fellow at the Creative Destruction Lab, the world’s largest AI venture accelerator. Tomi holds an MSc in Computer Engineering and an MBA from University of Toronto. Most importantly, Tomi is a husband and a father to two daughters.

    LinkedIn Github Google Research

  • Maksims Volkovs

    Co-founder & Head of Machine Learning, Layer 6
    Chief AI Scientist, TD Bank Group

    Maks is a co-founder of Layer 6 and the Chief AI Scientist at TD Bank Group. Maks received his PhD from the University of Toronto where he was part of the Machine Learning Group led by Geoffrey Hinton. He previously worked at Microsoft Research and at Credit Suisse in London, UK. Maks is the winner or medalist in multiple international machine learning competitions run by Kaggle (where his global peak ranking was #47 of over 75,000 data scientists), ACM RecSys (including winner of the 2017 and 2018 RecSys Challenge) and others. He has published over 15 papers and is co-inventor of 6 patents.

    LinkedIn Github Google Research Website

  • Laura Doktorova

    Chief Architect

    Laura is the Chief Architect at Layer 6. She has a solid track record of building efficient technologies. Prior to Layer 6, Laura worked at Milq Inc. as the Chief Architect for 4 years and at Blackberry in various engineering roles for 10 years. Laura is passionate about entrepreneurship. She is a co-founder of Bebedo whose Inquiro™ technology provides high-speed spatio-temporal search on large volumes of time and location sensitive data. Laura received her degree from Belarusian State University of Informatics and Radioelectronics.

    LinkedIn Github

  • Harry Braviner

    Machine Learning Scientist

    Harry is a Machine Learning Scientist at Layer 6. He is a mathematician and astrophysicist by training, and received his PhD, MMath and BA from Trinity College at the University of Cambridge. Before moving to Canada, Harry lived in London and worked for the research arm of a systematic hedge fund. His current research is focussed on reinforcement learning, and teaching machines to solve difficult control problems.

    LinkedIn Github

  • Guangwei Yu

    Machine Learning Scientist

    Guang is leading the Computer Vision team at Layer 6. The Team achieved 2nd place at the Google Landmark Retrieval Challenge and 4th place at the RSNA Pneumonia Detection Challenge in 2018. Guang is also the medalist in multiple international machine learning competitions including the only repeat winner of the ACM RecSys Challenge and others. Guang received his MScAC in Computer Science and BASc in Engineering Science from University of Toronto.

    LinkedIn Github Google Research Website

  • Saba Zuberi

    Machine Learning Scientist

    Saba is a Machine Learning Scientist at Layer 6. She has years of data science experience and has previously worked at TD Bank Group in Toronto and at TaskRabbit in Silicon Valley. Saba is a Physicist with a PhD degree from University of Toronto. She completed her Post Doc research at the Lawrence Berkeley National Laboratory in California.

    LinkedIn Github

  • Yichao Lu

    Machine Learning Scientist

    Yichao joined Layer 6 as a Machine Learning Scientist after a research internship at Layer 6 where he collaborated closely with the research fellows at the Vector Institute. Yichao received his MScAC degree in Computer Science at the University of Toronto. Before moving to Canada, Yichao lived in Shanghai and completed his bachelor’s degree from Fudan University.

    LinkedIn Github

Develop your career at Layer 6

What’s in a name? Deep learning is inspired by the human brain. The cortex has six layers of neurons, where the sixth layer integrates the deeper processing to produce an action.

We’re growing our team exclusively with people driven to be at the top of the game in machine learning.

View current openings

In the news

Obtenez des renseignements en français

Layer 6 s’est engagé à servir la communauté francophone au meilleur de nos capacités. Pour obtenir des renseignements en français sur notre équipe ou des opportunités de carrière, veuillez écrire à l’adresse info@layer6.ai.