場所：メインホール（1階）
招待講演1 [10月30日(月) 17:30 – 18:30]
Taco Cohen（Qualcomm AI Research）※オンライン講演
Geometric Algebra Transformers: A General-Purpose Architecture for Geometric Data
Problems involving geometric data arise in a variety of fields, including computer vision, robotics, chemistry, and physics. Such data can take numerous forms, such as points, direction vectors, planes, or transformations, but to date there is no single neural network architecture that can be applied to such a wide variety of geometric types while respecting their symmetries. In this work we introduce the Geometric Algebra Transformer (GATr), a general-purpose architecture for geometric data. GATr represents inputs, outputs, and hidden states in the projective geometric algebra, which offers an efficient 16-dimensional vector space representation of common geometric objects as well as operators acting on them. GATr is equivariant with respect to E(3): the symmetry group of 3D Euclidean space. As a transformer, GATr is scalable, expressive, and versatile. In various geometric problems, GATr shows strong improvements over non-geometric baselines.
招待講演2 [10月31日(火) 09:30 – 10:30]
Graham Neubig（Carnegie Mellon University）※オンライン講演
Prompt2Model: Generating Deployable Models from Natural Language Instructions
Large language models (LLMs) enable system builders today to create competent NLP systems through prompting, where they only need to describe the task in natural language and provide a few examples. However, in other ways, LLMs are a step backward from traditional special-purpose NLP models; they require extensive computational resources for deployment and can be gated behind APIs. In this talk, I will discuss Prompt2Model, a general-purpose method that takes a natural language task description like the prompts provided to LLMs, and uses it to train a special-purpose model that is conducive to deployment. This is done through a multi-step process of retrieval of existing datasets and pretrained models, dataset generation using LLMs, and supervised fine-tuning on these retrieved and generated datasets. I will describe the details of this process, as well as the larger implications for automating machine learning workflows.
招待講演3 [11月1日(水) 13:00 – 14:00]
Emtiyaz Khan（理化学研究所）
How to build machines that adapt quickly
Humans and animals have a natural ability to autonomously learn and quickly adapt to their surroundings. How can we design machines that do the same? In this talk, I will present Bayesian principles to bridge such gaps between humans and machines. I will discuss (1) the Bayesian learning rule to unify algorithms; (2) sensitivity analysis to understand and improve memory of the algorithms; and (3) new priors to enable quick adaptation. These ideas are unified in a new learning principle called the Bayes-duality principle, yielding new mechanisms for knowledge transfer in learning machines.
References:
- The Bayesian Learning Rule, (JMLR) M.E. Khan, H. Rue (arXiv)
- The Memory Perturbation Equation: Understanding Model’s Sensitivity to Data, (NeurIPS 2023) P. Nickl, L. Xu, D. Tailor, T. Möllenhoff, M.E. Khan
- Knowledge-Adaptation Priors, (NeurIPS 2021) M.E. Khan, S. Swaroop (arXiv)
- Continual Deep Learning by Functional Regularisation of Memorable Past (NeurIPS 2020) P. Pan*, S. Swaroop*, A. Immer, R. Eschenhagen, R. E. Turner, M.E. Khan (arXiv)
招待講演4 [11月1日(水) 16:40 – 17:40]
Petar Veličković（Google DeepMind & University of Cambridge）※オンライン講演
Decoupling the Input Graph and the Computational Graph: The Most Important Unsolved Problem in Graph Representation Learning
When deploying graph neural networks, we often make a seemingly innocent assumption: that the input graph we are given is the ground-truth. However, as my talk will unpack, this is often not the case: even when the graphs are perfectly correct, they may be severely suboptimal for completing the task at hand. This will introduce us to a rich and vibrant area of graph rewiring, which is experiencing a renaissance in recent times. I will discuss some of the most representative works, including two of our own contributions (https://arxiv.org/abs/2210.02997, https://arxiv.org/abs/2306.03589), one of which won the Best Paper Award at the Graph Learning Frontiers Workshop at NeurIPS’22.