Unlock AI Capabilities On-Premises

How are enterprises unlocking powerful AI capabilities on-premises? 🏠

‍

At the Six Five Summit, host Patrick Moorhead is joined by Rohan Grover, Senior Director of Product, Google Distributed Cloud, for a riveting discussion on on-premises strategies for Sovereign AI. As industries continue to be reshaped by generative AI, the need for on-premises solutions has skyrocketed, addressing critical business imperatives and regulatory, latency, survivability, and bandwidth requirements.

‍

Key takeaways include:

‍

🔹Accelerating On-Premises AI Adoption: Explore the macro trends fueling the rapid adoption of AI on-premises, driven by regulatory mandates and the imperative for internal innovation within enterprises.

‍

🔹Enterprise Strategies for On-Prem AI: Understand the core strategies enterprises are embracing to meet their on-premises AI needs, emphasizing unified infrastructure, flexible model choice, and overcoming significant industry challenges.

‍

🔹Industry Pioneers in On-Premises AI: Learn how sectors like financial services, retail, and telecommunications are leading the charge in leveraging on-premises AI for dramatically enhanced operational efficiency.

‍

🔹Google Cloud's Sovereign AI Approach: Gain deep insight into Google Cloud's distinctive strategy for Sovereign AI, offering customers unparalleled control, choice, and security through solutions like Google Cloud Data Boundary, Google Cloud Dedicated, and Google Cloud Air-Gapped.

‍

Learn more about Gemini at Google Cloud.

‍

Watch the full video at Six Five Media, and be sure to subscribe to our YouTube channel, so you never miss an episode.

‍

Or listen to the audio here:

Patrick Moorhead: The Six Five Summit, it's back. And unsurprisingly, we're focused on AI. And listen, we love infrastructure, we love chips, we love the build out, we love agents. This theme of this show this year is all about getting value out of that. And you know, we're talking about something that is special in my heart and that is essentially what, what we call in Moor Insights & Strategy. The hybrid cloud. Innovation can happen anywhere, right? It can happen in the public cloud, it can happen in the private cloud, it can happen on the industrial edge. And we are going to talk about how we light up these capabilities. And it is a pleasure of mine to introduce Rohan, who essentially runs product for Google Distributed cloud. Welcome to the show, Rohan.

‍

Rohan Grover: Thank you for having me here, Patrick.

‍

Patrick Moorhead: Yeah, and you know, I made the bet probably 10 years ago as a company. I was a big believer in the public cloud and all the benefits from it. But I was like, you know this, it would be interesting, really interesting to have a cloud model that was on premises and you have delivered it. Thank you, thank you very much. But let's drive in, let's kind of go high level here. What are some of the macro trends, more importantly, the business imperatives that are accelerating the adoption of AI, and in particular AI on premises.

‍

Rohan Grover: Absolutely. So we know that gen AI is having an impact everywhere, whether it is use cases like coding, customer care, improving employee productivity, AI and generative AI in particular is changing every business operation and approach that we have seen. But for a lot of organizations, especially those dealing with sensitive data, with regulations and compliance, or even organizations that require extremely low latency, this Genai revolution is kind of a little bit of feels a little distant because they can't really leverage public cloud for the reasons that we just talked about. And they have the same use cases that we just mentioned for everybody else. And we want them to be able to leverage the power of gen AI but do it in their own data centers, in their own premises. On premises.

‍

Patrick Moorhead: Yeah. And what I, what I really appreciate is you're giving customers choice. You want to run it in the big cloud, you want to run it in the private cloud, it's your choice. And I also saw some disconnected versions as well that are, that are, that are very, very interesting. So let's talk. I get in front of a lot of CIOs, CTOs and CXOs and you know, always cataloging the, hey, what are the impediments to scaling AI. I'm curious, from your point of view, what are you seeing? What are the complexities? What are some of the challenges that they're sharing with you?

‍

Rohan Grover: Yeah, and I'm going to speak now specifically on challenges for on Prem AI. Okay, start off with the fact that deploying on premises AI is expensive, time consuming and complex. And why is that? It's primarily because a, you're likely dealing with data sovereignty and regulatory compliance. Stringent rules based on the country that you're in can absolutely stifle AI adoption and really impact the customer experience. Then you have the complexity of AI infrastructure. As you said, this is a hybrid world and customers have to figure out by themselves which AI apps can they deploy on prem, what technologies can be used in cloud, what can be done in a hybrid way. It's extremely complicated and it's not obvious what works where. And lastly, let's also acknowledge kind of some of the cloud exclusive frontier models. Most gen AI vendors out there have made their models available and accessible only through the public cloud, making on premises deployment a huge challenge. And Google took a different approach. Google Cloud took a different approach. We announced the availability of basically the best of Google Cloud AI on Prem in April at our Google Next conference. And this is basically including our best in class large language model Gemini 2.5 that will be made available and delivered on prem using our Google Distributed Cloud portfolio.

‍

Patrick Moorhead: Yeah, I was there, I was in the seat. I wrote, you know, I have a full team of analysts. But that was, that was a note, that was a note that I took. And then I went to Dell Tech World and I saw it again. It was super, I was super, super exciting. So where are you seeing the highest levels of interest? You know we talked about, I think you had mentioned, you know, security latency or maybe there's just control that, that, that people want or, or there are certain regulations. Where are you seeing the pockets of interest the most?

‍

Rohan Grover: Yeah, and before I talk about it, I do want to give a call out to some of the partnerships that you, that you just mentioned. So Google is working with Nvidia. In fact, Jensen was the one who did the announcement on this at Google Next and we're thankful about that. Dell is a huge partner so we're working with industry leaders to make sure that we can bring this AI revolution on Prem. And then coming back to your question, I think we're seeing this across the board in multiple different industries. Let's start with maybe retail and quick serve restaurants. Right. They have use cases around adopting vision AI and conversational AI to make their customer interactions more personalized and efficient. They also want to make in store processes more efficient. So we're working with some of the largest kind of quick serve restaurants around the world to deploy AI on Prem and for them it's less a regulatory challenge, it's more a latency and survivability issue where they have to make sure that their restaurant or retail store stays up and running even if connectivity to the public cloud goes down. 

‍

And as you know, many of these restaurants are in maybe remote locations where the Internet connectivity is not always the best. So that's one. In public sector we are seeing huge traction again for compliance reasons, whether it's a public safety use case like emergency responding or its use case around sensitive financial data. We have a deployment in Luxembourg as an example where the financial regulator is deploying on prem AI to look at financial fraud detection use cases. And another example is manufacturing. There is assembly line worker safety that AI can help with with vision AI or process efficiency in figuring, figuring out how to improve semiconductor yields with predictive AI tools. So basically we're seeing this across the board and really customers are getting a huge value from this on prem AI technology.

‍

Patrick Moorhead: You know, I was in, I went to Davos this year to meet with folks. I was really trying to get a read on AI and investment. Everybody you know in the world comes there. But I wanted to get a read from folks primarily in Europe and there was a lot of talk about hey, I want my own sovereign cloud and I want it for AI. Is this applicable? Is your technology and offering applicable? I'll get this out to the sovereign AI cloud.

‍

Rohan Grover: Absolutely. And I'm going to pull back a little bit and talk about Google's approach to sovereignty and how the on Prem piece kind of ties into it. So we've been working on developing sovereign solutions for a decade or close to a decade.

‍

Patrick Moorhead: I didn't know that. Thank you for pointing that out.

‍

Rohan Grover: Yeah. And I know we've been iterating on this over the last almost 10 years. We recently announced kind of an updated view of our portfolio on sovereignty and I'm going to talk about that right now. So we believe that customers have the choice and we are providing them with the right level of controls and technologies to enable the level of sovereignty that they need. So on Google public Cloud we have Google Cloud data boundary. It allows them to control where in our public cloud regions the data is stored and processed. It allows them to store and manage their encryption keys and those encryption Keys are always in their control, which can help customers meet specific data access and control requirements. So that's one sovereign solution that we offer. The other one that, and specifically in Europe is called Google Cloud Connected, where we work with a local partner and we basically deploy Google Cloud in an independent dedicated region. 

‍

As an example, Google Cloud has partnered with Thales in the last four years to build out a trusted cloud solution. It's a partnership called Sons for Europe. And that's something that we've seen a lot, getting a lot of traction now and then last but not least, we do have a air gapped on prem option for a fully standalone solution that does not require connectivity to any external network or to the Internet. And this is really tailored for customers in the national security space, defense intelligence, where they have the strictest of data regulations and residency requirements. And it can be deployed at a relatively small scale or a very small scale. And this can be operated by a local partner, by Google themselves, if that doesn't break sovereignty rules or by the customer. Right. So we are offering basically a choice of sovereignty solutions depending on public sector requirements.

‍

Patrick Moorhead: Yeah. When I learned about the air gap solution, I said to myself, wow, this has got to get Google out of that comfort zone and was probably very difficult to pull off. Now I think I heard you said you were working on this for 10 years, but it is really is really good to see and again, all about choice. Choice is, is, is important and I think just the fact that you're, you're offering these things is a, is a, a really good message that you're sending to folks that, you know, quite frankly, you can do it like you want to do it, you want to do it in, in the public cloud. Great. You want to do it on prem, we can do that too. So. And I think you've been making pretty massive infrastructure investments across cloud regions, across zones, network edge locations, and pops. And I am amazed too at your lane cable underneath the sea to increase that performance between data centers.

‍

Rohan Grover: Yeah, and thank you for calling that out. Right. At this point, our infrastructure footprint has scaled massively. We are now available in 42 public cloud regions. We have 127 zones, 200 plus network edge locations, and 33 subsea cable investments. And that basically spans global infrastructure. And I believe that we are uniquely differentiated in this space.

‍

Patrick Moorhead: Yeah, that's, you do actually have the largest data state. When I, you know, I put the Google Cloud plus the consumer stuff stuff in there that I know a lot of people don't, don't, don't don't talk about. So hey, I want to talk about something pretty, pretty exciting to me. Looking, looking out over the next five to 10 years. Not asking you to, you know, pre announce what you're doing here, but more about what are the trends and the capabilities that you're monitoring and looking at for on premises AI.

‍

Rohan Grover: Yeah. So I think let's start with people think about when they talk about AI, they typically talk about models, but models are just one part of the overall AI stack. Google is bringing out the best models in the industry. Whether it's a foundational large language model like Gemini 2.5 or there are task specific models. So we're going to constantly innovate and come out with the best in class models with some of the largest context windows. Gemini 2.5 for example, has context windows that are the highest in the industry, going from 1 million to 2 million. And that just allows customers to process a whole lot of data in a single query. But we're going to see these models continuously learn and evolve and continue to get better over time. This is clearly an area that we are focused on, but outside of models, it's really the AI platform that we're really excited about. Google has created best in class serving, fine tuning and training platforms. You will likely take some of these models that we build and then you will fine tune them to make them work better in your environment, in your industry. So you're going to look at how to refine their understanding, improve their performance and adapt their behavior. 

‍

And when it comes to on Prem, you're going to use your local data to train this model based on your use case. So that makes it tremendously exciting. This allows customers to kind of operate with greater autonomy and really the sovereign use cases we talked about. And beyond models and the AI platform, it's the agentic AI applications that are really going to be differentiated. Ultimately, models and AI platforms are a way for customers to create agentic AI tools. Sometimes we will provide agentic AI applications directly to customers on Prem. In other cases they will build their own tools. And a really interesting example of an agentic AI tool is our recently announced Agent Space Search on Google Distributed Cloud. Now think about the problems with search. I mean actually let's think about Google. Like Google is good at many things, but what made Google Google was Google Search. So think about bringing the power of Google Search on Prem so that customers can really search through their own enterprise data. And today that enterprise data is completely siloed with multiple tools across teams. And we are bringing the power of Google search with large language models into on PREM deployments and this is one of the agentic AI tools that I am personally really excited about and when we have shared this with customers they've been super excited.

‍

Patrick Moorhead: Yeah, I'm still waiting for Vertex AI. No, I'm just kidding.

‍

Rohan Grover: Well, we do have some vertex AI capabilities. So as part of the AI platform a few of our key vertex AI capabilities are available on prem, including things like language translation from 200 plus languages as well as speech to text. So we do have some elements of our vertex AI platform on PREM as well.

‍

Patrick Moorhead: Yeah, I appreciate you sharing with us what the future might look like and when I step back and gosh, I'm leaving next two weeks from now I'm going to be with a room full of CIOs and they just keep perpetually telling me I'd like some choice but wouldn't it be great if I could closely partner with somebody who's very strong in the public cloud and I'll call it the, the on premises AI cloud that I can, I can bank on. And what you're pull pulling together here I will say and as an analyst I always need to watch how I say it or my excitement but I, I really like what you're, what you're putting together here.

‍

Rohan Grover: Thank you Patrick. We appreciate that. We want to bring the best of Google Cloud AI to our On Prem customers and we're going to continue to keep innovating across the stack.

‍

Patrick Moorhead: Yeah, exciting. Thanks for coming on again.

‍

Rohan Grover: Thank you so much.

‍

Patrick Moorhead: Yep. So thanks for joining us here in the Intelligent Edge Spotlight of The Six Five Summit 2025. Stay connected with us and our social media platforms and check out the new website sixfivemedia.com, stick around, we've got more AI insights to come. Thanks a lot.

Disclaimer: The Six Five Summit is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded, and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.

‍