About Courses that I have audited during Fall, 2022: CS294V and AMCS 343.

---

During my Fall Semester of 2022 at KAUST CEMSE/CS I audited two courses, that have connections to my research and my future.

Firstly, I’d like to mention that as a third-year Ph.D. student, I don’t have obligations to pass courses formally, however having the opportunity to audit courses is a great opportunity to think about methods in Distributed Machine Learning/AI problems from different angles and it’s useful for looking for new connections of Federated Learning with existing fields.

---

System Architecture and Performance, CS294V with Prof. Mootaz Elnozahy

The course System Architecture and Performance CS294V is taught at KAUST by Prof.Mootaz Elnozahy. Research areas of Prof. Mootaz Elnozahy include Systems, Distributed systems, Resilient computing, Security, System architecture, Performance, and Parallel Processing.

Professor is a Fellow of IEEE, a Fellow of the US National Academy of Inventors, and a Fellow of the AAIA. In the past, he was Professor at CMU, Senior Manager at IBM Research, Adjunct Professor at UT Austin, Dean at KAUST, Special advisor to the president at KAUST, and Visiting scholar at Bell Labs, Bellcore, and Meta (Facebook).

The most important outcome for me of the class CS294V is a new way of thinking about complex systems. I have obtained knowledge that performance is not about the parts or even the sum of the parts. It is about trying to master complexity.

Many performance problems come from focusing on one aspect of the system without understanding the interactions among system components.

The course covers Amdahl’s law and performance metrics; Introduction to the modern CPU; Pipelining and branch prediction; Pipeline performance and compiler support; Virtual memory and TLB; Memory architecture and caching; Cache consistency and coherence; Scalable Cache consistency and coherence; Synchronization Support.

In my opinion, the course is useful for people in CS who is interested in creating software systems in industry or academia that behave nicely with existing hardware. It covers, but not going into EE aspects of implementing electronic components of computing and memory. In my opinion, this course augments knowledge that can be obtained from Performance Engineering course 6.172 at MIT in a very nice way. For people who do have not big Systems CS background I highly recommend first looking into Performance Engineering course 6.172 at MIT first.

One of the lessons from the class is that It is not always the best technology that wins, but the companies that understand the hardware and software together often make sound decisions commercially.

---

Fast Solvers for Large Systems of Equations, AMCS 343 with Prof. Gabriel Wittum

The course Fast Solvers for Large Systems of Equations, AMCS 343 is taught by prof. Gabriel Wittum. Professor Gabriel Wittum’s research focuses on a general approach to modeling, simulation of problems from empirical sciences, high-performance computing (HPC).

The course AMCS 343 covers the Modelling and Simulation aspects of linear systems driven by ODE/PDE in Physics and Math and the formalization of computational problems via discretization in form of linear systems. The course covers Direct Solvers, Classical Iterative Solvers, Multi Grid methods, and Robust Multi Grid methods.

Fast solvers like parallel adaptive multi-grid methods allow for application to complex realistic models. These methods and tools are applied to problem-solving in fields including computational fluid dynamics, environmental research, energy research, finance, neuroscience, pharmaceutical technology, and others.