The visited AI summer schools during summer 2021.

Several engaging summer schools that I have a chance to attend during the summer of 2021.

Regularization Methods for ML 2021

The school was led by Lorenzo Rosasco from June 21, 2021, to June 25, 2021. This course’s main goal was to understand various modeling aspects and some of the optimization and statistical principles behind intelligent systems. The main focus was on regularization techniques. Regularization methods allow treating a massive class of diverse approaches in a unified way while providing tools to design new ones. The course started with classical notions of smoothness, shrinkage, and margin. After that, the course began to cover state-of-the-art techniques based on geometry (aka manifold learning), sparsity, and various algorithms for feature selection, structured prediction, and multi-task learning.

Link to the summer school:

Attendance certificate of Konstantin Burlachenko: Link


The PRAIRIE/MIAI AI summer school comprises lectures by renowned experts in different areas of artificial intelligence. The third edition of this summer school will be held online from the 5th to July 9, 2021. It will include presentations on several topics, including computer vision, machine learning, natural language processing, robotics, and healthcare. The 2021 summer school will feature lecturers including Lourdes Agapito (UCL), Stéphanie Allassonnière (Univ. Paris), Francis Bach (Inria), Alejandrina Cristià (ENS), Andrew Davison (Imperial), Pascale Fung (HKUST), Arthur Gretton (UCL), Hannah Kerner (Univ. Maryland), Yann LeCun (Facebook / NYU), Catherine Nakalembe (Univ. Maryland), Cordelia Schmid (Inria / Google), Jean-Philippe Vert (Google/ Mines ParisTech).

Link to the summer school:

Attendance certificate of Konstantin Burlachenko: Link

Oxford ML Summer School-2021

The second Oxford machine learning summer school (OxML 2021) aims to provide its participants with best-in-class training on a broad range of advanced topics and developments in machine learning (ML) and deep learning (DL). The school will cover some of the most critical topics in ML/DL. The field is growing interested in Bayesian ML, representation learning, computer vision, natural language processing (NLP), reinforcement learning, causal ML, transfer learning, healthcare, and medicine.

Reading materials

  1. Gaussian Process for Machine Learning

  2. Taking the Human Out of the Loop: A Review of Bayesian Optimization

  3. High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning

  1. Inductive Biases for Deep Learning of Higher-Level Cognition
  2. Toward Causal Representation Learning
  3. A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
  4. Learning Neural Causal Models from Unknown Interventions
  5. Systematic Evaluation of Causal Discovery in Visual Model-Based Reinforcement Learning
  6. Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization

Bayesian ML

  1. Variational Inference: A Review for Statisticians
  2. Advances in Variational Inference
  3. Tutorial on Variational Autoencoders
  4. Auto-Encoding Variational Bayes Probabilistic causal ML:
  5. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
  6. Geometric foundations of Deep Learning
  7. Causality: Models, Reasoning, and Inference, Cambridge, 2nd Ed., 2009. Chapters 1-3.
  8. Elements of Causal Inference: Foundations and Learning Algorithms, MIT Press, 2017. Chapters 1-6.
  1. Pattern Recognition and Machine Learning, Introduction to GPs, pages 303 - 320.
  2. Gaussian Processes for Machine Learning (free download)
  1. Attention Is All You Need
  2. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  3. BEHRT: Transformer for Electronic Health Records
  4. RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism
  1. Deep neural network models for computational histopathology: A survey
  2. Deep learning in histopathology: the path to the clinic Computational histopathology papers covered in Lea Goetz’s lecture:
  3. Clinical-grade computational pathology using weakly supervised deep learning on the whole slide images.
  4. Closing the translation gap: AI applications in digital pathology.
  5. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer
  6. Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images
  7. Pathology GAN
  8. Self-Supervision Closes the Gap Between Weak and Strong Supervision in Histology
  9. Capturing Cellular Topology in Multi-Gigapixel Pathology Images
  10. Data-efficient and weakly supervised computational pathology on whole-slide images

Multiple-Instance Learning:

  1. Multiple-Instance Learning for Medical Image and Video Analysis
  2. Attention is all you need
  3. GANs: Generative Adversarial Networks
  4. Segmentation: U-Net: Convolutional Networks for Biomedical Image Segmentation
  5. Contrastive Learning: A Simple Framework for Contrastive Learning of Visual Representations

Causal ML, Interpretability in ML:

  1. AutoML: powering the new human-machine learning ecosystem
  2. From black boxes to white boxes
  3. From real-world patient data to individualized treatment effects using machine learning: Current and future methods to address underlying challenges
  4. Bayesian Inference of Individualized Treatment Effects using Multi-task Gaussian Processes
  5. Validating Causal Inference Models via Influence Functions
  6. Estimating Counterfactual Treatment Outcomes over Time through Adversarially Balanced Representations
  1. “Our World in Data: Emissions by Sector” - context on the sources of GHG emissions
  2. “Oil in the Cloud” (recommended) - a worthwhile read on some of the negative uses of ML.
  3. “Tackling Climate Change with Machine Learning” (optional) - David Rolnick’s talk will be centered on this paper

Link to the summer school:

Attendance certificate of Konstantin Burlachenko: Link

Conference and summer school “Optimization without Borders.”

HSE University, MIPT, and Sirius University invite you to join the Conference ‘Optimization without Borders’. The Conference is dedicated to the 65th anniversary of Prof. Yurii Nesterov and the 50th anniversary of Prof. Vladimir Protasov.

Link to the summer school:

The conference program:

Written on October 24, 2021