Powering the New Industrial Revolution with AI and Sustainable Cooling Solutions - Six Five Media

We can’t underestimate the sustainability 🍃 question when it comes to the computing and storage demands of the AI industrial revolution. Host David Nicholson is joined by Lenovo's Matt Ziegler, Director of ISG Neptune and Sustainability, and NVIDIA's Mike Sabotta, Solutions Architect. They share their thoughts on the explosive growth of AI and the critical role of liquid cooling in ensuring efficient and sustainable data centers.

‍

Their discussion covers:

‍

• AI's Energy Demands: AI is driving an unprecedented surge in power consumption
• Liquid Cooling to the Rescue: The partnership between Lenovo and NVIDIA to drive innovative solutions that offer significant energy savings and a reduced carbon footprint
• Impact on AI Infrastructure: The role of Lenovo Neptune Liquid Cooling and NVIDIA's technologies in meeting the energy demands of advanced AI applications and infrastructure
• The Long Game: NVIDIA's mission to change the world through AI innovation and its search for strategic partnerships
• Moving Towards Sustainability: The impact of the innovation partnership on sustainability and technological advancement, including their GB200 announcement earlier this year.
  
  

‍

Learn more at Lenovo and NVIDIA.

‍

Watch the video below at Six Five Media and be sure to subscribe to our YouTube channel, so you never miss an episode.

‍

Or listen to the audio here:

Disclaimer: Six Five Media is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.

‍

David Nicholson: Welcome to this episode of Six Five In the Booth. I'm Dave Nicholson with The Futurum Group and Six Five Media, and I'm joined by two special guests to have a conversation about something, dare I say that is extremely cool. First, we have Matt Ziegler, Director of ISG Neptune, and Sustainability at Lenovo. Welcome, Matt. We also have Mike Sabotta, who's a Solutions Architect with NVIDIA. Mike, why don't you kick us off with a conversation about AI. Set the stage for us. What's going on in the world of AI as far as NVIDIA is concerned?

‍

Mike Sabotta: AI is the next Industrial Revolution. We are manufacturing tokens, so we are needing efficiency at every level. So if you look back over the last decade when we had done Kepler GPUs versus Blackwell today, there is 100,000 times difference in energy required to do those tokens. That lowering of energy is making it possible for people to be driving these demands for AI, whether it's material sciences, whether it's drug discovery, whether it's recommender systems, all of these things are requiring energy and efficient chips and acceleration. We are doing accelerations at every level, and this is pulling the solution together into a very dense packaging environment, and that density is increasing the energy density. It is very energy efficient, but it is dense. So to transport that away, we are now needing water. We are leaning hard on partners like Lenovo who have a decade plus of experience delivering warm water cooled solutions to the world.

‍

David Nicholson: So what could possibly go wrong, Mike when you start pumping water through electronics? Is there any possible issue that could arise there that you might be able to imagine?

‍

Mike Sabotta: Water is a wonderful transport. Chips are a wonderful creator of energy or consumer of energy, and they just go better together. We just need to make sure that we are doing the right things at a system level, making it robust engineering, minimizing the points of failure. So if you were to think about your car, your car has things like a front bumper and a seatbelt and an airbag, those are all things to help make it safe. System providers like Lenovo will be delivering those types of features for this world to make it safe.

‍

David Nicholson: So Matt, tell us about the history of Lenovo and liquid cooling. Is this something that you, is this a bandwagon you just jumped on this year when you heard about all this AI stuff, or have you been doing this for a while?

‍

Matt Ziegler: No, we've been actually doing it for a while. So I can give you guys some history. A lot of us started at IBM before Lenovo actually acquired us in 2014 when they acquired the System X business. Prior to that, we started looking at solutions around the 2008 timeframe, doing some prototypes around how can we liquid cool x86 based chips, that we were in the System x group. We had a legacy around mainframe, so we knew materials, we knew high quality like stainless steel, materials that are used in liquid cooling. We brought that knowledge over onto the System x side, and the reason that we did it wasn't because we had to, it was because we started to see areas in the world that were paying very high kilowatt-hour cost. So their energy costs were extremely high, and a lot of folks were going, how could we reduce unnecessary components?

‍

And the first thing we targeted was air movement. So if there's fans and there's air conditioning and you're making it, you're putting energy into just making cold air and moving cold air. So the customers and us, we looked and said, well, if we don't need that, maybe we can become really energy efficient. And so that's really where it started. And so you didn't see it become mainstream at all until we ran into a lot of problems that NVIDIA was in. How can I increase the transistor density? What does that happen to the power profile? What does that happen to heat generation? Well, we had already been doing it for energy efficiency reasons, so why not do it to actually solve thermal issues that we were seeing inside? So our first commercial system was in 2012, and it was the fourth-largest supercomputer by speed in the world when we first launched it. So the first liquid cooled x86 system was actually one of the largest supercomputers in the world out of LRZ in Germany.

‍

David Nicholson: So at first it was a bit of a nice to have in the sense that you're driving efficiency, you're driving down the amount of power. Do you have a ballpark number in terms of the efficiency advantages of liquid versus having those fans blowing? I think people underestimate just how much power those fans consume in an air-cooled model.

‍

Matt Ziegler: Yeah. And so over time, back when we first did it in 2012, you're looking about 4% of the system might've been fan power. Now you can allocate fan power, about a third of it. So as the silicon and the components, and the network cards, and memory, all of these things are now taking more and more and more in power. So you need more and more and more fan power. So it's hard to say, oh, I'm efficient here in 2016, and then in 2024, it's different. It's a different problem that we have to go solve. But what we're seeing today is anything that is air-cooling does require a high allocation of fan power. And what we found is that fan power is additive. 

‍

So you'll get the silicon performance, you'll get your workload, you'll get your jobs run, and then the fan power is on top of what the component, what the system is actually pulling. So it's gotten to the point where it's getting to be, to give you an idea, we're seeing fan power allocation be equal to or even more than what the silicon draws. So you're taking it away from the processors that NVIDIA makes that could be doing work, and you're putting it into components that do nothing but move air.

‍

David Nicholson: So Mike, to your point, yeah, I was going to say your point about kind of the factory and churning tokens, fans are not getting any of that work done for you, are they?

‍

Mike Sabotta: I want to say it's bigger than what Matt just said because what Matt just described is on the IT piece of the puzzle, there's the data centerpiece of the puzzle. So it is easy for technologists to sit around and say it's 40 plus percent energy reduction by going to a warm water cool. So one of the unique things that Lenovo has been driving is this high water temperature and NVIDIA is very much supporting that. So from a support of 40 degrees C water that reduces energy at the data center level and it is so much easier to cool it. So you can easily come up with a number of, let's say that we're dropping PUE by 0.2 or 0.3, and we're converting from fan power into pump power. So those two numbers easily add up to 40%, and the mileage is going to vary depending upon what kind of data center, but there are two parts of the equation. One is at the IT side and one is at the data center side. So when you look at energy required to create a token, it's the sum of those two.

‍

Matt Ziegler: I'm going to just finish with one quick point. You bring a really good point. I bet you a lot of folks think water cooling, I got to put cold water in, and going back to Mike's car analogy, it has a lot of things. One of the things you realize is the fluid that you're using to cool the engine block is hot. It has such thermal capacity. And so that's one of the things that we work with NVIDIA a lot, is how much do we actually have to cool, and if so, do we even? And then your energy savings. So that was a really good point.

‍

Mike Sabotta: There's also,-

‍

David Nicholson: It's interesting because we talk about power consumption needs increasing and being high. We talk about things getting hot and needing to be cooled, but what is the sort of state-of-the-art in terms of a 42U rack in a data center? How much power could it conceivably consume if you were to configure it with as dense and as powerful a configuration that NVIDIA can deliver? What might it need?

‍

Mike Sabotta: From an air-cooled standpoint, there are partners that are putting 60, 80 kilowatts of air-cooled things in a rack. Unfortunately, as Matt was describing, the fan power on that is significant. It's measurable, so. And when you have that fan power requirement, then there's also data center has to move it. They have to create 25 or 30C air to go and feed those. So the energy on that complete circle is what is important. So you can get dense with air cooling, but it's not efficient. Because this is a factory, an AI factory, the data center is the factory. You want to be efficient and warm water cooling is an important part of it. To make warm water actually requires less maintenance because it's a more simple radiator and fan versus a compressor and everything else. So there's been studies done on cost of maintenance. There are just so many layers of the solution that get to be optimized better when you're thinking about it as an AI factory.

‍

David Nicholson: Sure. So Matt, what are some of the other considerations? Yes, we need to keep these devices within operating tolerances. We want to be able to drive density and get all of that stuff done. But take a step back and tell me about what the overall environmental advantages are from what you're doing at Lenovo.

‍

Matt Ziegler: Right. And Mike definitely started hitting on this, and so when you look at a data center and you go read any article that's in there right now, you'll see that one of the problems that data centers are having is getting more power. So if they're maxed out on, they get a certain amount. Well, how do you carve that power up? And that's where you get into the PUE, the efficiency of where the energy's going. You want almost all of that energy to go as much as you can into the IT, into the tech. So you're doing work and that work is producing output for the entity, the research institute, whatever. So if you are now looking and saying, I've got, make some numbers like 10 megawatts and I'm only 60% efficient, you've got four megawatts going into that's not doing IT. So that's where we really want to start hitting.

‍

And Mike touched on it. These AC units pull a lot of power compared to a pump. The pump, I can remove so much more heat out of a system for a fraction of the energy using a pump. So that's where we really want, when we look at it from a holistic point of view. If I don't have to make cold air, I don't have to put power into blowing it around, I don't need the fan power inside the system, then I'm really golden. Now, there's one aspect also, is when we use warm water, when water is high going in, that means it's hotter coming out. We've now just captured energy. It was in a state that transformed into heat. Now we have heat energy. The next thing is then looking and saying, what could we do with heat energy?

‍

One of the customers we're working at, we are doing absorption chilling, where we're taking that warm water and actually using the energy to produce cold water that's needed for some of the other smaller air-cold aspects like storage and things like that that are not yet liquid cool. So we're using, as efficient the power coming in, how can we best use it? There's also building heating. Could you actually hook it into a transfer from the warm water that's redirected from the systems into your environment? So there's a lot of, from our IT aspect, we really do come at it from the data center down. On top of it, we also want to make sure that the products that we're putting in, like the NVIDIA components in there are operating at their best performance also. So we don't want to compromise that when we're thinking about energy efficiency too. Highest performance energy efficiency.

‍

David Nicholson: Yeah. It's worth mentioning that when we talk about 60 or 80 kilowatts of power, if you're an EV or a solar nerd like me, those numbers mean things. But look at it this way, a kilowatt of power is considered to be 1.3 horsepower. So you look at a data center rack and imagine that thing being the engine from a Subaru running at red line in every one of those racks. That's the kind of energy that's being consumed. It's absolutely remarkable. We have Grace Blackwell. Thank you very much, NVIDIA. Coming to a theater near you soon. Lenovo, you have something to talk about in the Grace Blackwell arena?

‍

Matt Ziegler: Yeah, we do. We just launched the SC777 with NVIDIA. I know Mike intimately was involved in working on that. There were daily calls. It was a very collaborative process of here was our solution. Here's how we want to package it. Here's how we want to cool it. What are the challenges from the NVIDIA side? What are the specs? So if you actually sat in one of these design meetings, you wouldn't know where NVIDIA was and where Lenovo was. It was one team. And that's one of the things I really like about working with partners like NVIDIA. They're not just throwing things over the fence and saying good luck. It really is a collaborative process because when we have a successful product, it really shine. It makes their stuff shine. Mike, I don't know if you want to add anything. You're the one that was daily on those calls at night. I know. For a long time.

‍

Mike Sabotta: I don't have anything to do on a Tuesday night. It's been awesome working with the team. Thermal distinguished engineer at Lenovo has been awesome. This process has been very collaborative and wonderful.

‍

Matt Ziegler: Yeah. I'm going to make one more comment because you made it at the beginning, so I'm going to circle back, is what is the, you got water, you got liquid, and you got electronics. You definitely don't want those to mix. Right. How many people have spilled something on their laptop and then they go, oh, no. So one of the things that we do a lot of research and innovation on is to make sure that doesn't happen. That we're getting the best thermal transfer, we're getting the highest thermal margin that we can on the components so that actually the NVIDIA silicon is running really ideally at a specific temperature. We worry about the flow rates, we worry about the materials.

‍

So we use copper in a lot of our cold plates where the liquid runs through with brazing. So we eliminate things like O-rings and points of failure that Mike pointed out, eliminating points of failure. We use high quality EPDM hose that you'll see that withstand high pressure. Those are treated to prevent corrosion. There's a lot of experience. You don't just fill water and it goes through. There's a lot of things to make sure that you don't have growth in the water, you don't have scaling, a lot of things like that, and they make the best silicon. We want to make sure that it runs as it should.

‍

David Nicholson: So by some estimates, something like 8% of global power generation is going to be dedicated towards AI in the near future. And if you double click on that and you look at electricity coming in and powering things like GPUs, if you've got 1000 watts going into a GPU, 1000 watts of power, you have to cool about the equivalent of 1000 watts of heat being generated. It's like a hairdryer practically. Right now between the two of your companies, we're at a good point where we can keep thermal considerations in check, but wouldn't it be great moving forward to be more efficient about using and reusing that energy moving forward? Let's start with you, Mike. Do you have any thoughts on what the future might hold in that regard?

‍

Mike Sabotta: This is a huge topic and a wonderful thing. In the past when we did air cooling, that energy, as you describe it, it's whether you want to call it watts or joules. Those joules of energy are being moved outside. We are basically getting them out of the data center and moving them outside. So those joules are just wasted. So when we had 40 degree air, it was very difficult to do anything with that air coming out of a server. Now that we have water, we can efficiently pump it someplace. So whether it is a district heating or aiding a manufacturing process or helping accelerate a biodigester, there are many examples of what we can do as a society with that energy. So recycling the energy is a wonderful thing.

‍

David Nicholson: So Matt, not expecting something shipping tomorrow in that regard from Lenovo. Don't want to put you on the spot, but do you have any thoughts on that subject? Right now we're talking about efficient cooling kind of in a vacuum, if you will. But what do you envision for the future of being able to recycle?

‍

Matt Ziegler: Yeah, I mean, Mike said it best with regards to capturing that heat. Now that you're able to basically absorb all of that energy into a liquid, what can you do with it? Let me kind of paint two scenarios. So if I'm cooling water, if I'm using energy to chill my water, and then the water going into a server is cold and it comes out maybe a little warmer, there's not a lot of energy that you can capture in there. That's why this is, you get kind of like twofold. If I don't use energy to chill the water, that means there's higher energy. So once we start talking about 60, 65 degree return temperature water, that actually now has something that can be usable. We started with a company in Germany, LRZ, a research institution. There's a lot of IT components that still aren't liquid cooled, like storage.

‍

So we were using that energy to actually go into what's called absorption chilling. It's actually taking the energy out of hot water and using that to create cold water. I know it sounds counterintuitive, but it is one of the coolest experiments to see. And so they're using the energy that they've already consumed out of the GPUs and the CPUs and all the IT components, the memory, everything we captured, and they're making cold water. Another thing that we've also worked on is buildings. So you can do heat exchange in a building. So if you've got a building that's hydraulically heated where you're using radiators or water, you can actually transfer that heat. So now you're saving energy because you're preheating all of your building hot water. So this is something that's very new. Now people have moved to liquid. Now they're saying, what else can we do? And we're just kind of on that cusp, and those are really good points.

‍

David Nicholson: So Lenovo partnered with NVIDIA, not only working on the latest generation of generative AI, but also keeping us warm. We appreciate that.

‍

Matt Ziegler: Yeah. I always make the joke that that's the way you get people to do work. They got to keep the computers going.

‍

David Nicholson: Perfect. Well, it sounds like a great partnership, a cool partnership, as I said from the outset. Power consumption, cooling, these are all absolutely critical factors that we're going to have to work through in the future, and it sounds like Lenovo's experience going back more than a decade is really paying off in the world of AI. Gentlemen, thank you very much for being with us today for Dave Nicholson. Thanks for joining us here in this episode of Six Five In the Booth.

‍