Ali Jadbabaie
Civil and Environmental Engineering Department, Laboratory for Information and Decision Systems and Institute for Data, Systems, and Society, MIT
IAS SEMINAR #10/2026
Turin / OGR / Sala Duomo / July 16, 2026 / 06:00 – 08:00 pm CEST
Attention Masks, Normalization, and Positional Encodings: A Systems theoretic Perspective on Large Language Models
EVENT&WEBINAR
Despite their remarkable capabilities, large language models remain largely black-box systems: we still lack a principled understanding of how their internal mechanisms route, transform, and mix information across layers. This talk presents a systems-theoretic perspective on core transformer mechanisms — attention masks, normalization, and positional encodings — arguing that they are best understood as interacting modules of a high-dimensional, nonlinear, time-varying dynamical system on graphs, where attention masks induce the underlying graph structure and information propagates through state-dependent weights, value transformations, and residual pathways.
We first examine oversmoothing and rank collapse as a stability and contraction phenomenon, showing that even adaptive, state-dependent attention can suffer exponential loss of expressive power when connectivity enforces sufficient mixing. We then show how attention masks and normalization jointly shape information flow, with different mask structures inducing different dynamical behaviors, and normalization reshaping the geometry of possible equilibria. Finally, we analyze positional encodings — decay masks and rotary embeddings — through a signal-processing lens, offering principled explanations for attention sinks, lost-in-the-middle effects, and long-context generalization. Together, these results recast foundation models as structured dynamical systems amenable to classical analysis, control, and design.
ALI JADBABAIE
Ali Jadbabaie is the JR East Professor and Head of the Department of Civil and Environmental Engineering at Massachusetts Institute of Technology (MIT), where he is also a core faculty in the Institute for Data, Systems, and Society (IDSS) and a Principal Investigator in the Laboratory for Information and Decision Systems. Previously, he served as the Director of the Sociotechnical Systems Research Center and as the Associate Director of IDSS as co-founder of its flagship PhD program in Social and Engineering Systems. He received a B.S. degree with High Honors in electrical engineering with a focus on control systems from Sharif University of Technology, an M.S. degree in electrical and computer engineering from the University of New Mexico, and a Ph.D. degree in control and dynamical systems from the California Institute of Technology. He was a Postdoctoral Scholar at Yale University before joining the faculty at the University of Pennsylvania, where he was subsequently promoted through the ranks and held the Alfred Fitler Moore Professorship in network science in the Department of Electrical and Systems Engineering. He is a recipient of a National Science Foundation Career Development Award, an US Office of Naval Research Young Investigator Award, the O. Hugo Schuck Best Paper Award from the American Automatic Control Council, and the George S. Axelby Best Paper Award from the IEEE Control Systems Society. He has been a senior author of several student best paper awards, in several conferences including the American Control Conference , IEEE Conference on Decision and Control, and IEEE International Conference on Acoustics, Speech, and Signal Processing . He is an IEEE fellow, and the recipient of a Vannevar Bush Fellowship from the Office of Secretary of Defense. He is a member of the Bush Fellows Research Study Group, as well as the National Academies Intelligence Science and Technology Advisory Group (ISTEG). His research interests are broadly focused on decision making, optimization and control, machine learning, network science and network economics, as well as quantitative and computational social science.
Register for In-Person Attendance
Joining us in person offers the opportunity for direct networking and interaction with both speakers and participants.
Please complete the form below to reserve your seat.
By clicking, you will be automatically registered for the in-person event.
Note: If you see a message saying you’re already subscribed, please ignore it — your in-person registration will still be confirmed.
Please note that, by attending, you implicitly authorize AI4I to film and photograph the audience during the seminar. By submitting your registration, you acknowledge and accept the AI4I Data Privacy Policy. Seats are limited: if you are unable to attend, we kindly ask you to inform us as soon as possible at ias@ai4i.it
LINK ZOOM
