SCAI Research Symposium

Toward Energy-Optimal AI

Generative AI adoption and its energy consumption are skyrocketing. Training a single frontier model can consume tens of GWh, and inference is rapidly outpacing training in aggregate energy demand. This surge is inflating operational costs, and power delivery is now the gating factor for bringing new GPU capacity online.

In this talk, I will introduce the ML.ENERGY Initiative, our effort to understand and curtail AI's runaway energy demands on three fronts. First, understanding where energy goes: I will present tools to precisely measure AI energy consumption and findings from benchmarking open-weight models across hardware and serving configurations via the ML.ENERGY Leaderboard. Second, optimizing energy use: I will describe how identifying computations on and off the critical path in distributed training enables precise GPU frequency control, saving energy on non-critical work without slowing down training. Third, exposing tradeoffs: I will present how co-optimizing static and dynamic energy through better kernel scheduling reveals the Pareto frontier between energy and performance, enabling practitioners to make informed deployment decisions under diverse constraints. All tools and systems are open-sourced through the Zeus project. I will conclude with open problems in building energy-optimal AI systems.

Mosharaf Chowdhury is an Associate Professor of Computer Science and Engineering at the University of Michigan, Ann Arbor, where he leads the SymbioticLab. His research focuses on making AI/ML workloads more efficient, with a particular emphasis on reducing their energy consumption through the ML Energy Initiative. Major open-source projects from his team include Infiniswap, the first scalable memory disaggregation solution; FedScale, a planetary-scale AI/ML platform; TPP, a tiered memory manager integrated into the Linux kernel (v5.18+); and Zeus, the first energy-optimal generative AI stack. Previously, Mosharaf invented the concept of coflows and was one of the original creators of Apache Spark. He has received numerous individual honors, including fellowships and paper awards from NSDI, OSDI, ATC, and MICRO.

AI Data Centers as Flexible Grid Assets

This talk explores how the rapid growth of AI is transforming data centers into major power consumers—and how, with the right technologies, data centers can become valuable grid resources instead. The talk will discuss emerging approaches to making AI workloads "power-flexible" and enabling data centers to dynamically adjust consumption in response to grid conditions while meeting performance constraints. Drawing from real-world deployments, the talk highlights how grid-interactive data centers can accelerate interconnection and support a more resilient and affordable power grid.

Professor Ayse Coskun is the Chief Scientist at Emerald AI, a startup focused on enabling power flexibility in data centers. She is also a full professor in the Electrical and Computer Engineering Department at Boston University, where she leads the Center for Information and Systems Engineering and serves as Associate Dean for Research at the College of Engineering (currently on leave from BU).

Prof. Coskun is widely recognized as a leading academic authority on orchestrating data center power demand in response to power grid needs. Her broader research applies AI and machine learning to optimize cloud and high-performance computing systems. She has received many honors for her contributions, including the Ernest Kuh Award for energy-efficient system-level design and an IBM Faculty Award for applying AI-based methods in DevSecOps. Earlier in her career, Prof. Coskun worked in industry at Sun Microsystems (now Oracle). She recently served as Deputy Editor-in-Chief of IEEE Transactions on Computer-Aided Design and holds a PhD in Computer Science and Engineering from the University of California, San Diego.

Massachusetts Green High Performance Computing Center

A talk introducing MGHPCC facilities, capabilities, and opportunities for research collaboration

John Goodhue is Executive Director of the Massachusetts Green High Performance Computing Center, an energy-efficient data center and consortium serving more than 20,000 researchers, students, and educators at Boston University, Harvard University, MIT, Northeastern University, the University of Massachusetts, and other institutions across the Northeast. His work focuses on regional and national collaboration around research-computing infrastructure, broadening access to advanced computing for researchers at small and mid-sized institutions, and developing a diverse next generation of computing professionals.

His research-infrastructure leadership includes the Northeast Storage Exchange, Open Storage Network, Northeast Big Data Hub, Eastern Regional Network, and Northeast Cyberteam. He also brings 30 years of industry experience in networking and high-performance computing, including technology leadership, engineering management, and general management roles at Cisco Systems and BBN Technologies, along with work on the management teams of several Boston-area startups. He holds a B.S. in Computer Science and Engineering from MIT.

The Sustainability Stack for AI at Scale: An Industry Perspective

AI infrastructure presents one of the single biggest opportunities of our time to align large-scale computing with sustainability goals — but realizing that opportunity requires scientific rigor, operational integration, and cross-industry collaboration that match the complexity of the underlying systems. From embodied carbon in accelerator chips to operational energy across globally distributed fleets, quantifying and reducing the environmental footprint of AI compute demands work across materials science, systems engineering, power systems, and policy. This talk describes how AWS approaches this challenge across interconnected layers. Measurement: building lifecycle assessment methodologies for AI hardware and operations, addressing data availability gaps, and working with service teams to quantify emissions from training and inference at the most granular level. Decarbonization of hardware and operations: making carbon metrics visible to service teams, integrating with financial and planning systems, co-owning reduction goals with infrastructure owners, and embedding sustainability into early-stage hardware and service design. Customer engagement: providing emissions reporting through public-facing tools and supporting responsible use of cloud and AI services. Cross-industry research and standardization: publishing methodology openly, contributing to Product Category Rule development, providing technical input to public policy teams on regulatory frameworks, and collaborating with partners across the full value chain — from manufacturers to academia to initiatives like SCAI — to advance the science of sustainable computing.

Fabio Grimaldi is a Senior Sustainability Scientist at AWS, where he leads the development of methodologies and large-scale models to track and improve the sustainability performance of AWS cloud services. He drives the adoption of sustainability metrics across AWS services, translating data into actionable insights that help AWS and its customers meet their sustainability goals. Fabio joined Amazon in 2022. He holds a PhD in Chemical Engineering from the University College London.

The Challenges of Measuring AI's Environmental Footprint: An Industrial Ecology Perspective

We can increasingly measure what AI systems consume directly, but it remains difficult to determine their lifecycle environmental impacts. For instance, operational energy and water use can often be measured with relatively direct activity data and established emissions factors. Yet for major AI infrastructure providers, much of the reported greenhouse-gas footprint appears to lie upstream—in semiconductor fabrication, server manufacturing, construction, and other capital goods or downstream (end-of-life treatment) —where data and estimates tend to be incomplete and uncertain. Beyond the own lifecycle of AI systems, their may arise major impacts, both positive and negative, on the broader economy and society that tended to be even harder to isolate and quantify, The result is a measurement asymmetry: some of the impacts that are easiest to quantify might not necessarily be the largest and most uncertain.

Industrial ecology, life-cycle assessment, and energy economics provide mature conceptual frameworks for addressing such issues. They distinguish attributional from consequential effects, embodied from operational impacts, and direct from indirect and economy-wide rebound. They also provide a disciplined way to ask a question often neglected in AI sustainability debates: how far should the system boundary expand before additional precision ceases to justify its cost?

From a public policy standpoint, California’s SB 253 is a first-of-a-kind policy going into effect in 2027 which requires large firms doing business in California to disclose Scope 1, 2, and eventually Scope 3 GHG emissions under standardized rules and phased third-party assurance. Such regulations necessitate consistency, transparency, and accountability in measuring and reporting lifecycle emissions of AI. But disclosure alone will not eliminate uncertainty where underlying supply-chain data remain modeled or incomplete. The central challenge, therefore, is not simply more measurement but distinguish what is directly observed from what is estimated, make uncertainty explicit, and design reporting systems that improve decision-relevant accuracy without creating a false impression of precision.

Deepak Rajagopal is a Professor in the UCLA Institute of the Environment and Sustainability and Dept. of Urban Planning in the UCLA Luskin School of Public Affairs. His fields of research include Industrial Ecology and Life cycle assessment, applied economic analysis of energy and environmental policies. He is also a faculty Scientist in the Energy Analysis Division at the Lawrence Berkeley National Laboratory. He has a Ph.D. in Energy and Resources from UC Berkeley, MS degrees in Ag. and Resource Economics (UC Berkeley), and Mechanical Engineering (U. of Maryland, College Park) and B.Tech. in Mechanical Engineering (Indian Institute. of Technology, Madras). He has been a post-doctoral researcher at the Energy Biosciences Institute, UC Berkeley and also worked as a Structural Engineer at United Technologies Research Center, E.Hartford, Connecticut.

Powering AI in a Thirsty World

The rapid growth of artificial intelligence (AI) is driving the construction of gigawatt-scale data centers, placing increasing demands on both power grids and water infrastructure. Yet power and water are tightly coupled: water-intensive cooling can compete for local water resources and create risks to data center resilience, while waterless cooling can increase electricity demand and further stress the local grid. This tradeoff is particularly significant on the hottest days of the year, when both power and water systems are under stress. Despite these interdependencies, power and water are often planned and managed separately, overlooking important opportunities for coordination.

This talk explores how responsible AI infrastructure design and operations can address coupled power-water challenges. I will discuss water-aware computing and cooling, coordination with power systems, and approaches for managing resource tradeoffs while strengthening infrastructure resilience and reducing impacts on surrounding communities.

Shaolei Ren is a Professor of Electrical and Computer Engineering at the University of California, Riverside. His research broadly focuses on developing modeling frameworks, algorithms, and empirical methodologies to address challenges at the intersection of AI, computing systems, and communities. He is a recipient of the U.S. National Science Foundation CAREER Award (2015) and several paper awards, including at ACM e-Energy (2024, 2016) and IEEE ICC (2016). He received his Ph.D. degree from the University of California, Los Angeles.

Direct and Indirect Energy Resources Enabling the Data Center as a Grid Asset

As data centers become increasingly integrated into the modern power grid, the demand for operational flexibility has never been greater. This talk explores the multi-faceted power and energy impacts of a flexible data center, specifically focusing on the complex interactions between direct energy resources, such as battery energy storage systems (BESS) and dynamic IT loads and indirect energy resources, such as water usage and flexible temperature interfaces. By examining these variables in concert, we can better understand how data centers can become grid assets, while maintaining the required availability of the IT workloads and constraining the use of the indirect energy resources.

Jeremy Rice, Ph.D., is a seasoned engineering leader with over two decades of experience spanning the "chip to chiller" stack. Currently, he serves as the Mechanical Systems Lead at Verrus LLC.

Prior to Verrus, Jeremy held a significant tenure at Google within their data center organization, where he focused on asset utilization, system simplification, and acted as a technical liaison between the IT and data center teams. He also brings extensive experience in IT-side hardware, having advanced the state of the art in both air and liquid cooling technologies.

Jeremy holds a Bachelor of Science and a Ph.D. in Mechanical Engineering from the University of Connecticut.

Flexible Data Centers Scheduling: Economic, Environmental, and Transmission Congestion Impacts

Simultaneously considering optimization of operating cost, greenhouse gas, and toxic emissions, this talk discusses a tri-objective, multi-period, power system-constrained framework to schedule flexible data center load. The framework Models data center power consumption as the sum of latency-critical and best-effort loads and considers the temporal flexibility of best-effort workload. The framework was implemented on standard power system test systems with data centers, and pareto fronts were obtained from the solutions. Trade-offs between different objectives are analyzed, and the impacts on electricity prices and system congestions were discussed.

Yuanrui Sang is an assistant professor in the Department of Electrical and Computer Engineering at the University of Massachusetts Amherst. Before joining UMass in 2024, she was an assistant professor at The University of Texas at El Paso, and she received her Ph.D. in electrical and computer engineering from The University of Utah in 2019. Her research interests include power system operation and planning, grid-enhancing technologies, and the integration of flexible load, such as data centers and electric vehicles, in power systems.

Data Centers, AI Workloads, and Efficiency: A Systems Perspective

The exponential growth of cloud computing has been a defining trend of our time, fueled by rapidly growing demands from online and data-intensive workloads. Despite the end of Denard scaling, the cloud's energy demand grew more slowly than expected over the past decade due to the aggressive implementation of energy-efficiency optimizations. However, the rise of AI workloads, which are often more resource-intensive than traditional cloud workloads, has led to rapid growth in data centers with power-hungry accelerators such as GPUs and TPUs, leading to a resurgence in the cloud's energy consumption and a strain on our electric grids.

In this talk, I will provide a systems perspective on the challenges and opportunities in enhancing the efficiency and sustainability of cloud platforms in the face of rising AI demand. I will discuss how resource management techniques such as workload shifting can enhance the efficiency of cloud platforms by exploiting the spatio-temporal variability in grid demand, energy availability, and electricity prices. I will present initial directions in making current computing systems grid-friendly and discuss approaches for navigating performance, efficiency, and cost tradeoffs that arise in their operations. I will end with several open research challenges that the research community needs to tackle to make AI-driven cloud platforms grid-friendly and ensure their continued growth.

Prashant Shenoy is currently a Distinguished Professor in the College of Information and Computer Sciences at the University of Massachusetts Amherst. He received the B.Tech degree in Computer Science and Engineering from the Indian Institute of Technology, Bombay and the M.S and Ph.D degrees in Computer Science from the University of Texas, Austin. His research interests lie in distributed systems and networking, with a recent emphasis on cloud and sustainable computing. He has been the recipient of several best paper awards at leading conferences, including two ACM Test of Time Awards. He is a fellow of the ACM, IEEE, AAAS, and AAIA.

AI Needs a Dose of Its Own Cure to Cut the Carbon. Let’s Do It!

As we ride the Cambrian explosion of AI, gains in the efficiency of resource use have become ever more important. They make the technology more accessible, enabling more models, features, products, and applications, increasing the value of AI. But as this community has pointed out, efficiency could backfire as a climate strategy: making AI more efficient could spur so much additional use that total consumption and absolute emissions might keep climbing. So if efficiency’s shadow twin, the availability of low carbon supply, is neglected, increasing value may come with rising environmental cost. After celebrating the progress this community has made on using resources efficiently, on carbon-aware computing, and on measuring embodied carbon, I will turn to increasing that low carbon supply of electricity and materials to build on, and I will share some of the work we are doing in this space. AI, so often seen as adding pressure against reaching net zero, can instead be a positive force to achieve it. Together, AI that makes resource use more efficient and AI that expands low carbon supply can drive a virtuous cycle, and this community can, and I will argue should, participate in both.

Karin Strauss is a Senior Principal Research Manager and Innovation Strategist at Microsoft Research and an Affiliate Professor in the Paul G. Allen School of Computer Science & Engineering at the University of Washington. Her work spans computer systems, synthetic biology and environmental sustainability, with research ranging from machine learning hardware and emerging memory technologies to biologically inspired computing. She is best known for pioneering DNA data storage systems, a project that received broad industry and media recognition. More recently, she has focused on making AI and IT infrastructure more sustainable.

Asset or Burden: Navigating the Community Impact of Data Centers

AI-driven data center growth imposes measurable externalities on local communities: noise, backup-generator emissions, water withdrawal, power quality degradation, and the potential for rising retail electricity prices. This talk characterizes what we can currently measure across each channel, then examines approaches for mitigating the community impacts and even providing benefits for community infrastructure using engineering, algorithmic, and policy levers such as siting, workload flexibility, energy and water storage, power-aware cooling, and policy levels.

Adam Wierman is the Carl F Braun Professor in the Department of Computing and Mathematical Sciences at Caltech. He received his Ph.D., M.Sc., and B.Sc. in Computer Science from Carnegie Mellon University. Adam’s research strives to make the networked systems that govern our world sustainable and resilient. He is best known for his work spearheading the design of algorithms for sustainable and community-centric data centers, including pioneering work on net-zero data centers, data center demand response, geographical load balancing, the public health impact of data centers, and the water usage of data centers. His work has seen significant industry adoption (e.g. through the startup Verrus). Additionally, he is well known for his work on heavy tails, including co-authoring a book on “The Fundamentals of Heavy Tails.” He is an ACM Fellow and an IEEE Fellow. He has received the ACM Sigmetrics Rising Star award, the ACM Sigmetrics Test of Time award, the IEEE INFOCOM Test of Time award, the IEEE Communications Society William R. Bennett Prize, the Caltech IDEA Advocate award, the Caltech GSC Excellence in Mentoring award, multiple teaching awards, and is a co-author of papers that have received “best paper” awards at conferences across computer science, energy systems, and operations research.

Rethinking Storage for Sustainable AI: From Models to Generated Data

The rapid growth of AI is creating a new storage sustainability challenge. Modern AI systems produce and retain enormous amounts of data—from billions of model checkpoints and fine-tuned variants to an ever-growing volume of AI-generated content. Yet today’s storage systems largely treat these objects as conventional byte streams, ignoring the rich structure and semantics introduced by AI workloads.

In this talk, I will present our recent work on rethinking storage systems for AI data. I will first discuss ZipLLM, which exploits relationships among models and combines model-aware compression with deduplication to substantially reduce the footprint of large model repositories. I will then present TensorDex, which pushes this idea further by treating tensors, rather than files or models, as first-class storage objects and exploiting relationships among tensors across an entire model ecosystem. Finally, I will discuss LatentStore, which revisits a more fundamental question for AI-generated data: do we need to store the generated object at all? By storing compact model-native representations and reconstructing data on demand, LatentStore trades increasingly inexpensive computation for reductions in long-term storage.

Together, these systems illustrate a broader opportunity: rather than applying traditional storage techniques directly to rapidly growing AI data, we can redesign the storage stack around the structure, semantics, and regenerability of AI workloads. I will conclude with a broader vision for sustainable AI storage, where computation and storage are jointly optimized to reduce the growing resource and environmental footprint of AI.

Juncheng Yang is an Assistant Professor in Harvard John A. Paulson School of Engineering and Applied Sciences. His research interests broadly cover the efficiency, performance, reliability, and sustainability of large-scale data systems and machine learning systems.

Juncheng's works have received best paper awards or honorable mention at VLDB'26, VALUETOOLS'24, NSDI'24, NSDI'21, SOSP'21, and SYSTOR'16. Juncheng was a Facebook Fellow, recognized as a Rising Star in machine learning and systems, and a Google Cloud Research Innovator. His dissertation on designing efficient and scalable cache management systems received the CMU SCS Doctoral Dissertation Award and the ACM SIGOPS Dennis M. Ritchie Doctoral Dissertation Award.

His works have been widely adopted. For example, S3-FIFO and SIEVE are adopted for production at hundreds of companies with more than 60 open-source libraries and packages in 18 programming languages. Moreover, his group maintains libCacheSim, the most popular cache simulation library, and freeinference, a free LLM inference service.

Resilient AI Infrastructure: From the GPU to the Grid

Modern AI systems run on massive, costly infrastructure that must operate under three kinds of change: workload changes, as agentic request rates, output lengths, and tool-calling times vary continuously; infrastructure interruptions, as GPU and network failures, preemptions, and maintenance repeatedly halt training at scale; and power changes, as available grid power fluctuates. In this talk, I will present our recent work on resilient AI infrastructure that adapts to all three. For workload changes, we use internal LLM signals to schedule inference requests, from standalone inferences to agentic workflows, cutting latency and improving efficiency. For infrastructure interruptions, we introduce TrainMover, a resilient LLM training runtime which leverages elastic and standby machines to handle interruptions with minimal downtime. For power changes, we started the Harvard Power and AI Initiative to rethink the coordination between the power grid and AI infrastructure—making AI workloads flexible and making grid planning matching such flexibility.

Minlan Yu is a Gordon McKay professor at the Harvard School of Engineering and Applied Science. She’s the assistant director of the SRC/DARPA JUMP 2.0 ACE Center for Evolvable Computing, and the co-director for the Harvard power and AI initiative. She received her B.A. in computer science and mathematics from Peking University and her M.A. and PhD in computer science from Princeton University. She received the ACM-W rising star award, NSF CAREER award, and ACM SIGCOMM doctoral dissertation award. She served as PC co-chair for SIGCOMM, NSDI, HotNets, and several other conferences and workshops.