Google Is Putting AI Agents Inside the Pentagon’s Systems. Here’s How They Keep Them in Line.

Government agencies could soon have 100 AI agents operating on behalf of every employee, each interacting with systems that were never designed for that scale. The challenge isn't just managing the infrastructure demand. As agents gain access to sensitive data, mission-critical applications, and systems of record, the consequences of a single agent operating outside its guardrails become far more serious. Scaling agentic AI across government means rethinking capacity, security, and control before autonomous systems become embedded in everyday operations.

Returning to the Google Public Sector Summit in Washington, D.C., for his second consecutive year of coverage, Patrick Moorhead sits down with Chris Hein, Technical Director and Field CTO at Google Public Sector. Hein explains how agencies can scale agents safely, what owning the stack from TPUs to applications means for government CIOs, and why he sees the constituent experience as AI’s greatest opportunity to make an impact.

For agency technology leaders, scaling comes down to capacity, cost control, and security working together. Google's own TPUs provide capacity during a global chip crunch, smart routing matches each task to the right model, and an agentic defense layer applies zero-trust principles to agents the same way they apply to the human workforce.

Key Takeaways:

🔹 Agencies are moving from chatbots to system-wide agents. Useful agents need access to data and tools, so Google Public Sector focuses on building them into agencies' systems of record with clear assurances about what each agent can and can't do.

🔹 Owning the stack solves capacity. Google's TPUs give agencies access to inference capacity when chips are scarce, enough to support the Department of War's GenAI.mil rollout to more than 3 million service members. Further up the stack, Hein argues data systems must be ready for a surge of agent connections, and Google offers open-source variants and third-party models to reduce lock-in concerns.

🔹 Smart routing keeps costs in check. Gemini can route work behind the scenes, with an orchestrator running on a top model and worker agents on Flash or Flash-Lite models. Agencies can reserve the most capable models for power users, run open-weight models like Gemma, and avoid paying for an always-on environment.

🔹 Zero trust now covers the agentic workforce. Building on the BeyondCorp approach Google adopted after a 2009 attack, Hein says agents represent a scaled version of familiar threats. Defenses must work at machine speed and use context-aware controls that recognize each agent's identity and block access that doesn't fit.

🔹 Better constituent experiences are the biggest opportunity. Hein wants government services to beat what people get in their consumer lives, with multimodal AI helping them fill out and understand forms for taxes, licenses, and new businesses. He also sees AI lifting the administrative burden on under-resourced caseworkers.

Agencies that pair scalable infrastructure with agent-level security can deliver government services that earn back public trust.

Watch the full video at sixfivemedia.com, and subscribe to our YouTube channel so you never miss an episode.

Learn more about Google Public Sector at publicsector.google.

Disclaimer: Six Five Media is for information and entertainment purposes only. Over the course of this video, we may discuss companies that are publicly traded, and we may reference their equity share prices. Nothing discussed during this webcast should be considered investment advice or a recommendation to buy or sell any security. We are not investment advisors, and you should not rely on this content as financial advice. Six Five Media collaborates with technology companies and industry leaders to produce research-driven interviews and multimedia programming for enterprise technology audiences.

Chris Hein:

You know, there was a time not that long ago that government had the best technology in the world. Right. We should get back to that. I want to make it so that the constituents' experience of government is better than what they get in their day-to-day life.

‍

Patrick Moorhead: 

The Six Five is on the road. I'm Patrick Moorhead. We are in Washington, D.C. at Google Cloud's Public Sector Summit. It's my second year here. This year's show was great, and the theme was all about from mission to moonshot, really balancing what people are trying to get done, organizations trying to get done, but really leaning in and being aggressive and trying to get even more done. No longer are we talking about, oh, we're going to do pilots and maybe take them into production now. It's hard, don't get me wrong, but really what the conversation is, is how do we scale this, right? How do we take AI and agents to scale? And for the second year here, I've got Chris Hine here, the CTO, Technical Director of Google Cloud Public Sector. Welcome to the show.

‍

Chris Hein: 

Thank you, Patrick. Really happy to be here.

‍

Patrick Moorhead: 

Yeah, you know, I really have the advantage as an analyst being here for the second year, because I can compare it to what you talked about in the first. And I'll admit, you know, I'm impressed with the way you have moved stuff, moved stuff forward. But it's crazy, things have changed so much. Yes, the good news is, is there were a lot of pilots done. a lot of AI in production, but now it's really time to scale and scale in a safe way. And if anything, you know, got people looking in, it was a lot of the agentic farm animals leaving the cages and going on to do something. And I'm sure that struck fear and concern inside of Washington, but what are you doing about it, Chris?

‍

Chris Hein: 

Trying to get the animals back in the pen? No, I mean, I think a lot of what we're doing now is really trying to educate across these agencies on what does it start to look like to move beyond personal applications of chatbots. And so I think that we had a really successful year of a ton of people really starting to get their hands on it, getting a little bit better understanding of what it can do to help them be productive. But then where we want to start to see a lot more traction, and we're starting to see it really become real, is a lot more of these kind of system-wide applications of these agents. And that's really one of the things that we're focusing on, is what does it look like to do that, as you mentioned, safely and securely? Because it's one of those things where in order for it to be truly useful at the enterprise level, that means we've got to start to give it access to data. And we've got to give it access to tools. And that gets scary if it can get out of the pen and go wreak some havoc. And so what we've really been focusing in on is what does it look like for us to build these things out, make them truly useful, put them into the systems of record that matter to a government agency, but do it in such a way that there's assurances all the way through that we know what those agents are doing, we know what permissions they have, we know what permissions they don't have, and we're locking all of those correct doors but making it useful so that we're getting real work done.

‍

Patrick Moorhead: 

One thread that carried over from last year, and I think it's again, not just last year, maybe it goes back years inside a public sector, is your full stack capabilities. And my analyst firm, and we track all the way from your TPU on the silicon, a lot of homegrown, you do more than the TPU, all the way up to the application layer and pretty much everything in between. Talk about the benefit of that. Because it's funny, the naysayers might say, well, It's the full stack. I'm locked in. This is going to be a challenge. But I think we've seen, and I think a lot of Google public sector success, particularly in federal, has been because you have the full stack.

‍

Chris Hein: 

Part of it is, again, if you start at that basic layer of our infrastructure itself, and the fact that we do have our own tensor processing units, TPUs, and that's really solving for, you know, the whole world is going through this capacity crunch, and the fact that it is just so hard to get access to good chips that can do this kind of AI inference work. And so the fact that we have been building from the ground up on Google's own ecosystem does give us this capability set of saying, you know, when the Department of War wants to launch genai.mil with 3 million plus service members, I got you, right? I have enough capacity to make something like that happen. And so that's been really, really key across every element of government is, you know, it's great if you want to use AI, but if you can't actually get access, it doesn't mean anything to you, right? And so then as you traverse up that stack, the other things that become really important, and it's especially important as we move to this next phase, this agentic phase, is the fact that you have to have a custom-built harness for all these things to start to ride on. And I look at it a lot when I look at the agentic data cloud aspects of it, because I'm a recovering DBA from years back and have seen many, many a failure over time when you've got way too many connections coming in that you didn't expect. And so now I think about that with agents, and you're going to have, you know, 100x agents that you have current connections on a daily basis. Are your systems ready for that? And you have to be able to survive in this new world of, you know, every single person might be launching up to 100 agents at a time going after that. And so I think You continue up that, maybe one more step up that ladder that's been important to our agencies. It comes back to our first question, which is that security and that governance aspect. Because I think what we've seen as you look at the news of agents escaping onto the internet and doing these things, alignment is a very challenging problem when you think about it from an AI perspective. But it's important to say, you don't have to solve it entirely if you've got the right agentic defense layer. If you can actually build for that and say, look, I don't have to trust everything fully because I'm going to put the right guardrails in place to make these things happen. And so to your point, I do think folks will get a little bit concerned about, well, is that a lock in? But I would say we do offer open source variants of just about everything. We try to be open to lots of other model providers, too, so that you're not just coming into this ecosystem and just getting everything from Google, but that stack does provide you with the opportunity space to really be part of the Agenda Cara.

‍

Patrick Moorhead: 

Yeah, and that is unique. And by the way, in a previous life, I was actually related to product development and product management. And full stack and optionality is hard, right? But you guys have pulled it off to your credit. We're looking at scaling large language models and even agents to millions of users, right? And I'm curious, this cost, how are you managing to put the right workflow and the right sub-agent against the right model. Like everything doesn't need Gemini 4, it might need a flash variant if it's doing something very basic. Are you doing token routing, things like this?

‍

Chris Hein: 

Yeah, so the different sophistications of the different agencies that we work with all have a bit of a different need when it comes to how they want this to work, right? There's a lot that's built into Google Cloud and Gemini as an agent actually has a lot of capability sets now to do some smart routing without you even necessarily as an end user having any idea that things are happening under the hood. Sometimes you're using an orchestrator that's for Argon and it's got a lot of worker agents that are off in the flash models, right? But I think the other thing that we're really trying to do is we're trying to build an ecosystem so that our biggest, most complex users, they can start to put that together in the kind of pieces that they want and kind of have their budgets set up so that they've got their top power users that we need to make sure that we're allocating enough space for them so that they can get the highest intelligence models the whole time that they need access to them. But maybe some of those rote tasks, things that the Flash or even the Flashlight series is good for, fantastic. Then let's make sure that we're putting the boundaries in place to allow for that. We still have plenty of customers that are also still interested in running an open weights model. And that's great. We can run Gemma as a great model from us. But there's all these other options that allow you to really optimize and say, I want to have an always-on AI environment, but I don't want to pay for an always-on AI environment, if you know what I mean.

‍

Patrick Moorhead: 

Yeah, that's smart. It's really giving people pretty much what they want. And with the Adder for the open models, that's huge. It's so funny, each wave of computing, security is really important. You know what I'm saying? But each evolution, it just always surprises me. Now, it's like security's really important with agents, okay? I mean, it was always important, but it really changes the game here. It's really the foundation, because in the end, one element, as we move from chatbots to, I'll call it mission-critical applications, was the data that you were hitting. And then it's this lightning speed ability to export, if you had a rogue agent, or come in even unannounced and come in and do some pretty nefarious things. I mean, we've seen it with developmental models of people who couldn't keep the animals in the cage, right? So how have you evolved how you look at threat intelligence? and how your zero-trust model has evolved to keep up with the pre-AI era. I don't know when that was, a while ago, to the chatbot era, to now the agentic era, where we literally could have tens of thousands of agents, even per employee, based upon what happens.

‍

Chris Hein: 

You build on the good foundational work that has come before. And so, you know, from Google's perspective, all the way back when we wrote the first BeyondCorp, you know, white paper that kind of talked about our security practices after, you know, we had a hack that came out of China back in 2009. And we really had to change everything about how we thought about our own corporate infrastructure and how to protect it. And so that, you know, zero trust is a bit of a buzz term, but it is an important kind of concept that we start off from a foundational perspective. Because I think that the really important thing to think about is, you know, we're at this phase of we've gotten pretty good, right, of protecting ourselves from human threats. And a lot of what the agents do, because of how we've kind of built these things out, they're just a scaled version of that same threat. So you start to think about it from the perspective of, you know, instead of just there being some of the top hackers out there that are capable of penetrating these systems, you have to start to make the assumption that every bad guy has a team of really good hackers on their side. And so that starts to just change your dynamic a little bit of saying, my scale of defense has to go up, but the actual ways that I'm protecting are very similar. And so we're really able to reinforce a little bit of that. And then it also becomes the same kind of zero trust methodology, but for your agentic workforce, just the same as your human workforce. And so you're really looking at trying to say, you're going to need machine speed because the agents are moving faster than a human can through a network. But you're still looking for some of those context aware types of blockers where you're saying, it's weird that you're accessing now. I can see your SPIFFY token. I know what ID you are, what agent you are. You shouldn't be in this data at this time. So you're going to need to be able to react to that in ways that are much faster than what you would have done with a human workforce. But generally speaking, we can adopt those same paradigms. It's just reinforcing them at the scale that we're now at.

‍

Patrick Moorhead: 

Like any type of security threat, education has always been key. Because in the end, sometimes it's the human making the mistake. And while it'd be fun to say, oh, the machines will make no mistakes, they do what a human allows them to do. So that part of the education, some of these agent breakouts were just, they were giving unlimited access to every tool across any data they wanted. you wouldn't give a first-time employee, a first-year employee access to the ability to ACH and wire out money, okay? So part of this is, again, we're gonna need this continued re-education type of campaign that goes off it, and I mean, it's still gonna require humans, because machines are perfect in their imperfection as well. because they're made by humans. So, let's wrap up, okay, with, you know, I'm going to make you, I'm going to force you to choose your favorite child here, okay? So, there's a lot of noise in the AI market. Trust me. You know, I'm on the opposite end. I get bombarded by every credible AI vendor, every non-credible AI vendor, and… I'll ask you about which ones are which later. Yeah, exactly. We can do that. And you're definitely in the credible. I'm sure you were wondering. But it's like it's the barrage of stuff coming out. And then what that does is it confuses the end customer. So if you picked one specific area for Google public sector of where you can make the most impact, where you're the most differentiated for agencies, where would you steer them in there? maybe they're not customers of yours yet. Like what is the, it's funny, I always call this the, you'd have to be crazy not to buy this from Google Public Sector. And I only get one, huh? You know, one and a half. Listen, it's your format, it's my show. If you want to go two or three, go ahead. All right, all right, all right. But the first one has to be number one.

‍

Chris Hein: 

It's got to be meaningful, yeah, makes sense. I think that the thing that I instinctively started to reach toward as you were talking about that is I do think when we think about constituent experience of government. So interestingly, because it's a bit of a left field from some of the stuff we've talked about at the summit today, but it is a place where I might have the most excitement about what this could look like. because I think most people's experience of government tends to not fill them with joy. It doesn't spark joy for them. And I think that that's something that we can flip completely on its head. And I think that when we look at some of the ways that the Gemini has the ability to work in multimodality where whether it's taking a picture of my forms and then it can pull that information in, whether it's I can then converse with a form and ask it questions of like, what is it looking for here? And it understands me, it understands my context, and it can help me to respond and put that together. I think that there's this really sweet spot of let's start to really make it so that when someone has to, whether it's they're filing their taxes, getting a license, opening a small business, I think it's possible for us to start to have that experience of that even better than what they tend to get out of their consumer life. And I, you know, I would love to, you know, I got challenged on a question at a panel a while back. I mean, like, should government move that fast? Should they do that? And I'm like, you know, there was a time not that long ago that government had the best technology in the world. We should get back to that. I think that that's not an unreasonable ask. That's going to be my favorite child, so to speak, is I want to make it so that the constituents' experience of government is better than what they get in their day-to-day life. I think beyond that, I do care deeply and want to see then the caseworkers inside of government, that they work inside of these benefit systems that are so critical and they do so many things that are vital to the American public. And it's just the most under-resourced group you can imagine. And so if we can start to really relieve that administrative burden from them and make it so that they can actually do what they actually probably wanted to do when they joined the government to help people, that I get really excited about it as well. Because I do think the technology is there. Now we just got to get it into the right place, get it past the right security parameters, all those kinds of things we've already talked about. Because I think if we can do those two things well, People's trust in their own government is just gonna go through the roof.

‍

Patrick Moorhead: 

Yeah, the first one is a natural for Google, because I mean, you guys are planet-scale consumer, and you're a giant B2B and business-to-government capability as well. And the other thing I heard in there was the moonshot. We can do amazing big things for you. We're doing it for a lot of people, and you can too. And that connects the dots between mission to moonshot. And the second one is removing the barriers, the time it takes to do things and the steps and the processes to make stuff happen. I love to see some of the stuff out of the State Department, which is like, you know what, we're gonna prove your passport in like a day, okay? You're not gonna like… those little things that people in the public who are paying taxes for, but also relieving the burden of the administrator so they can manage as opposed to being the arbiters and owners of a billion if-then statements of stuff that needs to be done with paper and electronically. So I love those. Thanks for extending the two, Chris. I appreciate that. No, Chris, this has been a great conversation. and appreciate the success that you have. And, you know, it's interesting, it's kind of a noble cause that you have here. I mean, people paying tax dollars to come in and they want it done quicker and they want it done more cheaply and the administrators want to be able to do it easier and maybe even get some more job satisfaction out of that. I think that's good for everybody.

‍

Chris Hein:

Absolutely.

‍

Patrick Moorhead: 

Chris, thanks for the time and maybe I'll see you next year.

‍

Chris Hein: 

I would love that. Thank you, Patrick.

‍

Patrick Moorhead: 

All right, everybody, great conversation here. I just love the reality check here, and you're going to get that with a field CTO. Like Chris, we love having Chris on the show. Check out all of the content from Google Cloud and the prior summit. Also watch the other two videos from this summit. Take care, everybody.

‍