Turning AI Data Centers Into Grid Allies with Emerald AI
Varun Sivaram is Founder and CEO of Emerald AI, a company building software that makes AI data centers power flexible. As AI data centers become one of the fastest-growing sources of electricity demand, grid constraints are emerging as a critical bottleneck for compute deployment.
In this episode, the conversation focuses on why power availability — not GPUs — is increasingly the limiting factor for AI. Data centers concentrate massive electrical loads in specific locations, creating grid stress, long interconnection delays, and rising electricity costs for surrounding communities. Traditional grid expansion alone is too slow to meet near-term AI demand.
Emerald AI’s response is to treat AI data centers as flexible loads rather than fixed ones. Its software coordinates compute with grid conditions by shifting workloads across time, geography, and on-site energy resources like batteries. The episode walks through real-world demonstrations, including a published field trial showing a 25% power reduction during grid stress without breaking compute performance. The discussion frames flexible load as one of the fastest ways to unlock power for AI while improving grid stability.
Episode recorded on Feb 2, 2026 (Published on Feb 10, 2026)
In this episode, we cover:
(1:36) What Emerald AI is and how it works
(6:41) Varun’s background and why he founded Emerald
(10:59) Emerald’s software for power-flexible data centers
(19:04) The three types of flexibility: temporal, spatial, and resource
(23:29) How much control customers give Emerald
(28:20) Coordinating compute with on-site energy like batteries
(31:27) Off-grid vs. grid-connected data centers
(35:39) Why exiting the grid creates political and systemic risk
(37:12) Emerald AI’s open roles
Links:
Varun Sivaram on LinkedIn: https://www.linkedin.com/in/varunsivaram
Emerald AI: https://www.emeraldai.co/
-
[Cody Simms] (0:00 - 1:20)
Today on Inevitable, our guest is Dr. Varun Sivaram, founder and CEO of Emerald AI. Emerald builds software that makes AI data centers power flexible. They can dial electricity use up or down when the grid is stressed without breaking compute performance.
We talk about Emerald as a kind of cloud scheduler for power and what they have already demonstrated in the field. We get specific about what Emerald does inside a data center and the three levers they use, shifting workloads across time, shifting workloads across geography, and coordinating compute with on-site energy resources like batteries. We close on why flexible load may be one of the fastest ways to unlock more power for AI.
From MCJ, I'm Cody Simms, and this is Inevitable. Climate change is inevitable. It's already here, but so are the solutions shaping our future.
Join us every week to learn from experts and entrepreneurs about the transition of energy and industry. Varun, welcome to the show.
[Varun Sivaram] (1:20 - 1:21)
Cody, thanks for having me.
[Cody Simms] (1:22 - 1:36)
I'm excited to dive in with you all about load flexibility with respect to data centers. I think that's sort of the headline of Emerald AI, but maybe take a minute and just describe Emerald AI to all of us. We're going to dive into there a ton.
[Varun Sivaram] (1:36 - 2:27)
Emerald AI seeks to make AI data centers, what NVIDIA calls AI factories, power flexible. That means we want these AI data centers to modulate their power consumption on command. So when the grid is stressed out and the AI data center is able to reduce its power consumption by a certain amount, 25% for a few hours, well, that data center is then going to be able to connect to the power grid more quickly, access a larger power interconnection, help support the power grid stability, and oh, by the way, potentially reduce power bills for the neighboring communities.
We believe that flexibility should be one of these core capabilities that every data center can have because data centers, in addition to being the biggest new user of electricity for power grids, could actually be the grid's greatest ally.
[Cody Simms] (2:28 - 3:08)
You know, I think of this moment in time, it's funny, you know, AI, obviously the future and everything, but it actually, to me, hearkens back to the 1950s, right? The post-war boom when for the first time in the US, we saw this, you know, big load forecast increase as people were adding air conditionings and people were adding washers and dryers. I think the difference there is that those loads were pretty diffuse across the United States, whereas with these data center loads, they're often very heavy, very pointed, and as a result can add incredible stress on certain parts of the grid at once.
Is that the right framing for folks, for me?
[Varun Sivaram] (3:09 - 5:28)
Well, first of all, it's always helpful to think through historical analogies. And so, as you've said correctly, Cody, like America for the last couple of decades has had flat electricity demand, and now we're starting to see this ramp up in demand. And so we think to ourselves, when was the last time we saw something like this?
And you're right, the 1950s and 60s, we saw air conditioners and other forms of industrial demand drive up electricity load year after year, and our electricity system had to expand to meet that load. So in that sense, there's a historical analogy. We've done this before, we moved mountains to make sure that everybody could be powered by an air conditioner, and now we need to move mountains to make sure that AI data centers can also be powered, the single most important economically productive electric load in human history.
But I think the historical analogy probably ends there. You picked out one reason why the analogy doesn't work, because air conditioners were diffuse, but AI data centers are particularly concentrated. Now, to be clear, maybe AI data centers going forward will evolve where there are many more edge or smaller inference focused data centers providing low latency services, and perhaps it will look more diffuse and less concentrated in the form of what we see today, one gigawatt or more campuses, mega campuses focused on AI train.
But nevertheless, setting that aside, there are a bunch of other reasons why I think the historical analogy can kind of get in the way of preparing for what is an unprecedented surge. First is, I personally don't think AI is what we call a normal technology, a technology like any other. I don't think when Elon Musk says that what we need to do is secure more power than America uses in entirety today, or secure more power than the world uses in its entirety today to power compute.
I don't think he's exaggerating. I don't think that the demand surge from AI data centers will resemble any demand surge from any technology before, because AI is likely to use more energy than everything else combined by orders of magnitude. And the second reason that the analogy probably fails is AI is a fundamentally different energy user than any other technology that's come before.
AI can move its computations around the country and around the world at the speed of light. You can't do that with air conditioners. You can't even do that with electric vehicles, right? You can't really decide to charge your car in Indiana one night and Virginia the next night, depending on where the grid conditions.
[Cody Simms] (5:28 - 5:38)
There's some latency need with AI, right? Real edge inference loads need to be near the query. It's really the training loads that in theory could live anywhere, right?
[Varun Sivaram] (5:39 - 6:15)
There are. And we should talk through some of those constraints, the constraints that limit what flexibility operations you can do. But just to close this point, we do ourselves a service by remembering, hey, in history, we have managed to meet large loads.
We do ourselves a disservice by saying we just need to do what we did in the fifties. What we're about to face is something unprecedented. And for as many people who talk about whether AI infrastructure is a bubble and whether it'll pop, I encourage folks to think about the opposite question.
Are we underestimating compute needs and energy needs? And are we therefore being too small minded in what sorts of changes we do?
[Cody Simms] (6:16 - 6:41)
Let's take a pause and just make sure folks know a little bit about you and your background and what led you to start this. And then we're going to dive into product. I want to really dig into product and how the Emerald service works.
But you've been on the show before. The show had a different name at the time when it was My Climate Journey, and you had a different job at the time when you were working at Orsted. Maybe share a little bit about some of the past work you have done that led you to the insights around starting Emerald AI.
[Varun Sivaram] (6:41 - 9:47)
I've spent the last 15 years in the energy space, building large scale energy infrastructure. But I will say, when I was an undergrad at Stanford, I had a left to right decision, study AI, which was one of my passions, or study energy and physics. I chose the latter, and that set me on one trajectory for the next 15 years.
I did my PhD in condensed matter physics, focused on solar technologies, and then worked on building large scale solar and other energy infrastructure around the world. I was at ReNew Power, India's biggest clean energy producer. I was their chief technology officer.
They're a publicly traded company. I was at Orsted, as you mentioned, the world's largest offshore wind producer, and I was their chief strategy technology officer. And we built multi-billion dollar pieces of energy infrastructure.
So most of my life was build big projects to produce more energy. And the big cognitive shift for myself was, well, can we build our way out of this? If AI is truly going to be this unprecedented energy user, it's certainly going to need far more energy generation.
But if we only build energy, I fear that we will face a twin dilemma. And we should talk a lot about these two twin problems. Problem number one is, can we build AI as fast as humanly possible?
And problem number two is, can we make energy affordable for everybody, even while AI eats the grid? I realized that just building our way out of this wouldn't work. That instead of just focusing on the supply side, where I'd been for the last 15 years, I also wanted to focus on the demand side.
So Emerald AI was born out of this shift. Actually, about 10 years ago, I wrote a book called Taming the Sun, the future of solar energy. And I had to kind of throw away half a chapter about wouldn't it be nice if data centers could move their computations to where solar energy was abundant.
In fact, you could follow the sun over the course of 24 hours. That idea has kind of come full circle, where we're not quite doing that at Emerald, but it's a related concept where by adjusting demand, you can actually squeeze far more juice out of your existing energy system and solve those twin problems, the speed to power crisis and affordability crisis. So that's where I am.
About a year and a month ago, I founded Emerald AI. I was fortunate. I consider as co-founder Radical Ventures, the firm that originally seeded us to build this company.
And very quickly we built an impressive, and I'm so grateful for the syndicate of investors, whether it's Google's chief scientist, Jeff Dean, one of our angels, or Fei-Fei Li at Stanford, one of the founding professors of the AI revolution. NVIDIA was a crucial partner and investor. And ever since we have been working with partners and investors to build Emerald AI into what I consider to be an ecosystem play.
I don't really think we have competitors. Instead, we seek to be a picks and shovels play to enable everybody else to make money. We want NVIDIA to sell trillions of dollars more GPUs.
We want digital realty to build far more data centers. We want Oracle to be able to offer far more cloud services. We want everyone to make money by building AI much more quickly and squeezing more juice out of our existing electricity system.
[Cody Simms] (9:48 - 9:55)
And your primary customer is a data center operator. Is that the right way to think about how you direct the product to the market?
[Varun Sivaram] (9:56 - 10:59)
Exactly. We built an end-to-end software product. This end-to-end software product enables a data center to communicate with the power grid and be power flexible.
Now to be clear, our end-to-end product also involves functionality for the utility or the system operator to use in order to observe, auditor, and dispatch the flexible data centers on their network to modulate their power consumption. So we are that software layer that unites these two multi-trillion dollar networks, the multi-trillion dollar power grid and the multi-trillion dollar network of emerging AI data centers and fiber optic networks. Those two basically don't talk to each other today.
One of them, the AI data centers plugs into the other one, the power grid, and just operates independently of whatever the power grid needs. And the power grid seeks to always supply whatever that data center needs. We seek to be the intelligence that connects those two and acts as that interface layer so that the power grid and the data center infrastructure can be co-optimized and we get the best out of both infrastructure nodes.
[Cody Simms] (10:59 - 11:03)
From a product perspective, I believe the product is called Conductor. Is that right?
[Varun Sivaram] (11:03 - 11:31)
The product is indeed called Conductor. Cody, you're a product guy. You're probably also a product marketing guy.
So let me say this. I have put not a great deal of effort into the nomenclature. So forgive me because a lot of this came off the top of my head.
But yes, the platform is called Conductor because what we aim to do is conduct an orchestra of available resources and flexibility tools to achieve the power behavior that we want.
[Cody Simms] (11:32 - 12:13)
The way I think about it is hyperscalers from a cloud compute perspective have had scheduling for a while. They've built scheduling into the bones of AWS or whatever, Azure, name your favorite cloud service. And as I think about it, those mostly optimize for latency and cost, right?
They're trying to reduce latency as much as possible. They're trying to minimize cost to whoever's using their service provider. As I understand it, Emerald AI similarly is kind of a scheduler, but rather than optimizing for latency and cost, you're optimizing for power availability and grid stress.
And I assume there's some cost components to that as well for the data center. Am I thinking about that the right way?
[Varun Sivaram] (12:13 - 14:10)
You absolutely are. Let me just back up and say we stand on the shoulders of giants. Although we're a one-year-old startup, there have been these pioneers like Google who have been developing tools for very sophisticated orchestration of compute resources for goals such as how do you achieve 24-7 clean energy?
They've done a lot of great work on this. And power flexibility is one of those advances they've made. It's one of the reasons we were proud to invite Kate Brandt, Google's Chief Sustainability Officer, and Jeff Dean, their Chief Technology Officer, as our angel investors.
The point you made I think is absolutely right. There's already a lot of sophisticated work on making sure that within one of these two multi-trillion dollar infrastructure networks, the data center and fiber optic network, it's highly optimized. But there isn't a lot of work to co-optimize the power grid network and the data center and fiber optic network.
So as you correctly said, today if you're meta and you're running a service like Instagram and for whatever reason there's congestion in one of your data centers or you need to reroute traffic for one reason or another, they do that with an extremely sophisticated set of tools. That's because what they want to do is optimize the performance and uptime of their service for all the users who want to scroll Instagram while taking best advantage of all the server infrastructure they have, server utilization, and best utilizing their assets. But nobody today makes the kind of decision that, ah, today the power grid is facing stress or may face stress in the next few days, and those power lines are running at well below full utilization today, but tomorrow they might be running at full utilization.
And we should make decisions on what computations to run at what speed and in which locations based not only on whether the servers are available and the fiber optic network is available, but on whether the power line network is constrained and the power plants are hitting their capacity.
[Cody Simms] (14:11 - 14:45)
Cloud providers famously have said, hey, this big batch of a load that we need to manage, we're going to run it at 2 a.m. in the U.S. so that we're not congesting our network, so that we can enhance a bunch of data around all these images that users have submitted in the last 24 hours. And that is a known thing that cloud providers do. But like you said, what they're not thinking about today is the cost of power or the grid's availability of power to do that, because honestly, it's not been a very expensive process so far to just do that for basic Internet usage.
[Varun Sivaram] (14:46 - 18:08)
Let's back up there, because you mentioned that what we do might relate to the cost of power, which it actually really doesn't. And so I want to back up and just make sure your audience knows what it is we're talking about here. I brought up early on the twin problems.
One problem is you can't get data centers connected to power grids fast enough. The other problem is everybody seems to be paying larger rates, higher rates for power. In Columbus, Ohio, in 2025, the average household energy bill went up by $240, largely attributable to data centers coming to town.
Why does this happen? The reason this happens is because a data center comes to a particular community and says, I want to plug into this part of the electricity grid. The grid, the system operator, the utility running the study to interconnect them says, well, you're a 200 megawatt data center.
And on the worst moment of the summer here in Phoenix, Arizona, when a million air conditioners are running, we don't know if we can serve you at that moment. Might only be an hour, but we just don't know if we can serve you. So we're going to make you wait for seven years, and we're going to build out our network.
We might have to reconduct a power line here or build a substation or build it, you know, expand a peaking natural gas power plant's capacity so that we can serve you. And if that data center is completely inflexible, the grid's doing the right thing. Now, the result of this is you get fewer AI data centers built in America, which is bad for American competitiveness in this race with China, which has 400 gigawatts of spare capacity by 2030 to build AI.
And you raise everybody in Arizona, you raise everyone's power bills because everybody pays for that reconductoring of a line, the expansion of the grid network. So this is a bad outcome on all counts. But if that data center is willing to be a little bit flexible, that data center says, look, actually on that worst day of the year, we are actually willing to ramp down our capacity, our instantaneous power draw.
We won't do it for most of the year, 98, 99% of the year, leave us alone. But for that rare moment where you're really congested, we will help you out. Well, then you can connect the data center more quickly.
The utility may say, I can fit you on today's network right now. Come on in. And I don't have to increase everyone's bills.
I actually get to reduce them because everyone's bills comprise two components. One is the supply of energy, the fuel cost, for example, of natural gas. And the other is amortizing, paying down the cost of the fixed asset base.
The grid lines have already been built and that you're paying for it over the next 25 years. If the fixed asset base remains fixed, we're not building out more grid, or at least we're building it out more slowly. And the variable revenue that comes in increases because these data centers are paying more dollars for every kilowatt hour of jet energy they use.
Well, then there's more money in the system to pay for the fixed asset base and everybody, your grandmother's power bill goes down. It doesn't go up. Just to back out, this has nothing to do with the cost of power that really is irrelevant to a data center today.
Data center is using GPUs that are a hundred times more expensive to idle for an hour than the cost of the power that you would save. We don't care about the cost of power. One day, by the way, we will care about the cost of power.
That day may be in a year or three years when the cost of inference and the demand for inference rises so much that we really need the inference costs, which are now approaching the marginal cost of energy. We need the inference costs to be rationalized. And so maybe you'll want to save energy cost by computing at a time when energy is abundant and not computing at a time when energy is scarce.
But today that's not the problem we're solving.
[Cody Simms] (18:09 - 19:04)
And I hear that echoed with almost every conversation I have with a hyperscaler or a neocloud or whatever. It's time to power that matters. How do I get this thing online as fast as possible?
And all the constraints you mentioned, I think are what are preventing that time to power right now, which is the challenges that the utilities have being able to green light a new data center project because of those external constraints. As I understand your product, it's really trying to optimize against kind of three big issues, right? There's temporal flexibility, which is we've talked a little bit about time shifting loads, moving them around to different times of day, helping data centers do that.
There's spatial flexibility, which is can you move loads from one geography to another? And then there's resource flexibility, I think, which is really how do you orchestrate the on-site resources of a data center? Do you want to unpack those?
Those are pulled right off your website, so you may be able to articulate them a little bit better than I just did.
[Varun Sivaram] (19:04 - 19:55)
As you correctly said, there are these three kind of domains of flexibility that we seek to harness. The first is temporal flexibility. So you might have certain computations that you can just slow down or pause for a certain amount of time.
Middle of the summer in Phoenix, Arizona, all the air conditioners are running and you need to give the grid three hours of relief by reducing your power consumption by 25%. This was the first demo we did that was published in the scientific journal Nature with the Electric Power Research Institute and Oracle. And what we showcased is, well, the one way to do it is to look at all the jobs that are running and see, are there some of these jobs that customers are okay slowing down?
Maybe you're fine-tuning a model and that model fine-tuning doesn't have to happen right this second. Just like you mentioned, the operation of enhancing the quality of images is another batchable operation that you can probably run overnight.
[Cody Simms] (19:55 - 20:16)
Before you go into the other two, let's dig into this one because you actually have published results from this demo in Phoenix where I think you created a 25% reduction in grid stress for a few hours. Walk me through, like I'm a data center operator, what triggers do I see and what do I do as a result of Emerald saying, hey, there's an optimization you can make here.
[Varun Sivaram] (20:16 - 23:11)
In that particular trial, we partnered with our friends over at Amperon, Sean Kelly, the CEO is our advisor. They run fantastic grid forecasting services. So thanks to Amperon, we kind of knew a week in advance when the stress event was going to happen.
So if you're a data center operator and within your four walls, you've got as your tenant, a cloud provider, let's say it's Oracle and Oracle has some customers. Emerald software is going to receive some forecasting data and relay that data and say, you know what, this is going to happen a week from now, two days from now, et cetera, and give you a little bit of heads up so that you and your customers can prepare. Now, let's say you've done some of the preparatory work and you have at least a couple workloads running that are possible.
When the event happens and the grid says, oh my goodness, I'm reaching the stressful point, they're going to send a signal and our software Emerald is going to receive that signal. We have a module we call Grid Link. So Grid Link receives the signal and it says, okay, thanks utility for letting us know what you need right now is a 25% power reduction for three hours.
We're going to work on that. We're also going to communicate back to you that we've achieved that target by proving to you the auditable log of the telemetry so that you are very confident that we did what we promised you we would do. And then to the data center now Emerald provides software throughout the stack to provide them visualization tools, dashboards, monitoring simulations.
We have a simulator that simulates, hey, how would you actually achieve this? Some compute customers may want everything. In addition to our orchestration software that Conductor offers that actually slows some of the jobs down.
Other customers may say the visualizations are really helpful and I'll just go ahead and do my scheduling and orchestration myself. There's like a menu of options for customers. In this particular demo, we went ahead and did everything.
So there were a range of jobs that were running and the chief AI officer of Databricks, which was the simulated customer in that case, ran us through a bunch of representative workloads that were running and the representative priority levels for each of those fine tuning or pre-training jobs or inference jobs. Some of these inference jobs were very high priority or very latency sensitive, like don't slow them down, don't pause them. And some of the other jobs, fine tuning jobs were lower priority and could be slowed down.
So in that particular demo, we slowed down the ones, you know, down to an acceptable threshold for the customer. We didn't slow down the ones that weren't allowed to be slowed down. We had various operations involved.
This is all public. We stopped some jobs. We reallocated the number of GPUs working on other jobs.
And in other cases, we actually slowed down the actual clock frequency of some of the GPUs. So there are many different levers we had available to us. And the net effect was that we kept the jobs running according to the customer specifications.
So the customer was happy. That's the number one priority. We also exactly met what the grid needed, which was a 25% reduction for three hours.
And we were able to prove it all to both sides, to the compute customer and to the grid. And that's all been published.
[Cody Simms] (23:11 - 23:29)
You're not running this with full autonomy, though, right? It sounds like you're working in partnership with the local operator. In this case, it sounded like Databricks was ultimately the one who needed to have the loads running at certain times or not running.
And so you're making recommendations to them and then they're implementing changes at the data center.
[Varun Sivaram] (23:29 - 24:17)
So it's a great question. Again, it's really up to the customer to determine what level of control they want. Now, there is no world in which Emerald knows anything at all about whatever it is you're doing.
We have no desire to know if you're training Deep Seek. We have no idea. We have no idea what your model weights are.
We have no idea, no ability to look inside your black box. But if the customer says, hey, we want Emerald to actually go ahead and orchestrate the scheduling of these workloads or go ahead and directly control the clock frequency of the GPUs, our software will go ahead and do it. If, as you said, what the customer prefers is recommendations so they can do it themselves, our software does that too.
It's kind of light, medium and heavy versions of our software. The goal is to give the customer as much comfort and choice as possible so that they can feel in control of the environment or provide as much control as they feel comfortable offering to the Emerald platform.
[Cody Simms] (24:18 - 24:45)
Super helpful. Thank you for the detour there, but I think providing some concreteness really, really helps at least me understand really what it is you're building and delivering to your customers. So that was an example of this temporal AI flexibility.
Let's go to the second sort of area that you're really trying to provide service, which is around spatial AI or sort of geographic workload shifting from a flexibility perspective. Explain that one a bit more.
[Varun Sivaram] (24:45 - 26:10)
AI data centers in general have this unique capability to move their economic activity around the country or the world on fiber optic networks at the speed of light. This is really rare and awesome. We should take advantage of it.
Let's say you've got a particular set of operations that you can't really pause. You don't really want to slow them down, but you could take the latency hit of moving them 500 miles away. It takes a couple of milliseconds.
Well, if you did that, then you might not degrade the customer experience in any way, but you would achieve the grid benefit you were seeking to achieve. Just today, the Electric Power Research Institute, EPRI, we're talking today on February 2nd. Just today, EPRI announced a series of new demonstrations after our big Phoenix demonstration, and Emerald's just delighted that we are partnering with them to execute many of these demonstrations.
One of them is spatially shifting workloads between two different locations, between Virginia and Chicago. I'm delighted to share that this actually works, that when Virginia has a grid event, it is possible to geo-shift workloads, in this case, inference queries. We had multiple inference clusters set up, and you're able to save power in one location to provide relief to the Virginia grid and move AI workloads over to the other grid, which is Chicago.
Delighted that EPRI's announced that that demonstration will be taking place, and we'll be delighted to share results when the time's right and EPRI has technically validated them.
[Cody Simms] (26:11 - 26:17)
Do you see this being more of an inference workload shift for doing this, at least for now, more so than a heavy, hard training load?
[Varun Sivaram] (26:18 - 26:29)
The first thing I'd say is I'd caution against saying there are only two buckets of AI workloads, inference and training. There's hundreds of different varieties of AI workloads, each with their own specific parameters for flexibility.
[Cody Simms] (26:29 - 26:32)
I have to keep something simple in my head because this is all very complex.
[Varun Sivaram] (26:33 - 27:48)
So this is a really nice way to think about it, that AI training often is not that time-sensitive, but it's data-heavy. It's hard to geo-shift it because there's so much data you've got to move. It's also a very synchronized process, and you try to set it up in one particular place and it's a little brittle.
You probably neither want to pause nor shift the largest training runs. They're so complex. But fine-tuning operations on that spectrum probably feel a little less mobile, but more plausible.
Inference has the opposite characteristics, at least serving inference. Batch inference, by the way, inference where it isn't that time-sensitive, and for example, you may be chemistry lab asking for protein models to be given to you overnight or next week. That sort of inference operation can be both paused and moved.
But there are serving inference operations. The classic one is, Cody, you're talking to ChatGPT and you're asking it a question. Hey, give me a sense check of whether this Varun guy is actually who he says he is.
Well, you probably want the answer right now so you can get on the podcast with me. You don't want to wait two hours. And so that's an example of an operation that is shiftable.
You move from one place to another because it's not that data-heavy. I just reroute your query. It's a few tokens.
[Cody Simms] (27:48 - 27:51)
An extra 500 milliseconds on the response there doesn't really matter.
[Varun Sivaram] (27:51 - 28:07)
And to be clear, 500 milliseconds is way longer than how long it takes to move your query from Virginia to Chicago. That's a good example of something that's not that data-heavy but is latency-sensitive and is a good case for geo-shifting rather than temporal pausing.
[Cody Simms] (28:07 - 28:20)
And let's hit the third sort of big bucket of product that you're rolling into Conductor, which is this notion of resource flexibility, which, as I understand it, is sort of the on-site energy plus compute orchestration capability.
[Varun Sivaram] (28:20 - 28:34)
Exactly right, Cody. The classic way to achieve flexibility at any load is to use an energy resource. You might have a battery, a fuel cell.
There's a ridiculous number of new data center projects that has behind-the-meter energy generation.
[Cody Simms] (28:35 - 28:41)
This is your Orsted background, like full force, I think, right, coming into play in terms of this side of the problem.
[Varun Sivaram] (28:42 - 30:16)
Exactly. Now, I believe that if you try and only solve this issue of AI flexibility through energy resources, you'll probably fail. The reason you'll fail is you're treating AI as a black box. You're saying, hey, it's an energy user.
I know nothing else about it, and it's not controllable. And so I'm just going to expensively and inefficiently throw a bunch of energy resources at that problem. And it treats AI as if it is a conglomeration of electric vehicles or a steel factory or whatever it is.
But I think that resource flexibility, these energy resources, in conjunction with computational flexibility, with temporal and spatial compute flexibility, can be a really powerful tool. If we have some batteries on-site, we should take advantage of those batteries to provide some of the flex and also rely on the compute to provide the rest of the flex. And yes, Emerald provides the brain that intelligently dispatches on-site energy resources alongside the computational resources we have.
So put all those together, and AI suddenly becomes the most flexible energy user in history. I truly believe, Cody, that there's a paradigm shift we can achieve, which is today, AI is this villain. It's this painful energy user.
Grids are afraid of it. It shows up in town, it raises your power bill. And on the other hand, there's this vision I have of AI being the grid's greatest ally.
It shows up to town, it reduces communities' power bills, utilities depend on it to help stabilize their grid. And going forward, utilities will trip over themselves competing to connect that flexible data center instead of backing away from this inflexible new load that's kind of scary.
[Cody Simms] (30:17 - 30:52)
I love particularly the description you just had on the on-site resource allocation, because I think a lot of startups we talk to, a lot of solutions in the space really are anchoring in on that. And as you said, that doesn't necessarily contemplate the complexity of what's happening inside an AI load, which is where you get this geo-shifting and time-shifting capability that can also add to on-site resource changes. But it's not just, hey, we need power to run a bunch of heat pumps at this industrial facility.
It's not a pure energy equation. There are many other factors involved in these AI loads.
[Varun Sivaram] (30:52 - 30:57)
We got to stop treating AI as a black box. It is a living, breathing organism that uses energy.
[Cody Simms] (30:57 - 31:26)
When you think about on-site resource shifting, where my mind goes from an AI perspective is this clear trend of data centers looking to build off-grid or behind-the-meter power to power their operations with some ability to then sell excess capacity back to the grid. How does that trend play into what you're seeing with Emerald AI? And will those be a customer for you as well, if they want to operate that way primarily?
[Varun Sivaram] (31:27 - 34:10)
Let me wear two hats here. Hat number one is the Emerald AI hopefully becoming a profitable company hat, which is I think Emerald AI and our ability to make data centers flexible power users will be useful whether the data center is fully grid-connected, partially grid-connected and powered by behind-the-meter energy resources, or fully off-grid, or even in space where the energy is entirely off-grid. In all of these cases, being able to control the amount of energy you consume is very useful to make sure it matches up with generation.
In fact, it's even more useful when you don't have this massive stabilizer and called the grid, and you're really on your own. Well, you better have the capability to match up your load and your generation. So we'll do just fine no matter which paradigm emerges.
All of them will be our customers. Here's what I worry about though. America already has plenty of excess generation.
We have the ability to bring on 100 gigawatts of new data centers without building a single new generator or power line. There will be some additional infrastructure improvements, of course, that are important, but we have plenty of stranded generation on our existing power system, except for those rare peak load moments during the year in each local grid. So if the solution, again, is just build our way out of this, these data centers basically leave the grid, build their own generation, we're doubling down on the problem, which is we're already inefficiently using our existing system, and now we're going to inefficiently use a brand new system that we build without ever trying to use AI as an intelligent energy user to better use what we already have. So I do worry a little bit that the intuition in Silicon Valley is, man, this grid stuff sounds way too complicated.
So many regulations, 3,000 utilities, everybody's governed by a public utility regulator. This is so darn complicated. Can we just go to space?
Or if not that, can we just build these private grids that are entirely islanded from the power grid and build our own natural gas plants? And that may work for a time, but there are some serious problems with that. The first problem is it may work today when the willingness to pay for power cost is basically infinite, but it won't work in the long run when the marginal cost of intelligence falls to the cost of energy, and you really want the most efficient and cost effective energy setup possible.
The grid is a more efficient way of sourcing energy because you're aggregating it across millions of users than building your own private use network built with your own generation, your own backup generation, your highly redundant infrastructure. You will probably fail to make the most cost effective and reliable system. Problem number one is we'll just have less AI.
Problem number two is a political problem.
[Cody Simms] (34:10 - 34:35)
By the way, before you get into problem number two, what you just described to me is classic Silicon Valley, which is the status quo is just too complex. We're going to end around over here on the side and just build our own thing and it's going to eventually take over the system because it's going to grow faster than it and the gravity of it is going to suck all activity into it. I'm hearing you say that's not the most efficient outcome given what the grid can and does do today.
[Varun Sivaram] (34:35 - 35:35)
It's not the best outcome for humanity. Look, I take Elon Musk extremely seriously. You never bet against the guy.
He's going to build in space, more power generation and more data center capacity than all of America uses on earth, not just for data centers, but for power generally. He will succeed. Launch capacity over the next decade will increase by a factor of a hundred thousand.
He will succeed. The question is, should we only max out our space-based data center capacity or should we max out our earth-based data center capacity on top of maxing out our space-based data center capacity? And my answer is Emerald exists because we believe that this is not an impossible problem, that grids are complicated, regulations complicated, but we're going to get as many data centers built on earth as possible.
So they'll compliment the likely more expensive data centers that are going to get built in space. We'll probably end up needing both. And I think we do ourselves a disservice by not maxing out what we can build on earth.
[Cody Simms] (35:35 - 35:38)
Thank you. And then you were going to go into, I think, some of the policy realities as well.
[Varun Sivaram] (35:39 - 37:01)
The policy realities, look, electric power utilities are granted, awarded monopolies in the areas in which they operate. And we have over a century of electric power regulation and deregulation. It will not go over well to have Silicon Valley giants go ahead and just ditch this entire system and build out an entirely parallel energy infrastructure that neglects everything that the grid has to offer and what needs to be repaired in this grid with the single-minded pursuit of building largely more expensive AI.
So given that, I expect a major political battle. If you completely try and exit the grid, my best prediction, my best crystal ball is over the next decade, maybe 10% of new AI capacity will truly be built off grid or in space. 90% will be built on grid.
And so if you're betting on exiting the grid, you're probably betting against the preponderance of where compute capacity is going to be built. My exhortation to the field is let's solve this. We're better together.
Let's solve how our grids work. And by the way, build this beautiful set of software solutions for AI data centers to contribute to grid stability and take advantage of fiber optic connectivity. That is a parallel grid in some sense.
It's a parallel fiber optic, an optical network. Let's take advantage of that to strengthen our existing grid and not abandon it altogether.
[Cody Simms] (37:01 - 37:12)
Varun, thanks so much for joining today. Are there any last areas where you need help? Anything you want to let our listeners know about what you're working on or where folks could lean in if they're inspired to do so?
[Varun Sivaram] (37:12 - 37:34)
We are always looking for talent. And I'm proud we have an extraordinarily talented team, a lot of PhDs, a lot of AI experts, but we're looking for talent across the board, on the business side, on the operation side, on the technology side, on the product side. So please send us your recommendations or apply to come work with us.
We'd be delighted to have you.
[Cody Simms] (37:34 - 37:35)
Amazing. Varun, thanks for your time.
[Varun Sivaram] (37:36 - 37:37)
Cody, thank you so much.
[Cody Simms] (37:38 - 38:04)
Inevitable is an MCJ podcast. At MCJ, we back founders driving the transition of energy and industry and solving the inevitable impacts of climate change. If you'd like to learn more about MCJ, visit us at mcj.vc and subscribe to our weekly newsletter at newsletter.mcj.vc. Thanks and see you next episode.
