Scaling Agentic AI with Arm AGI CPU

A processor about 1.2 times faster per chip on average can deliver roughly twice the work per rack in Signal65's modeling, because power and space set the limit. That gap makes rack density the variable that decides how much agentic CPU work a data center can host.

Ryan Shrout, President at Signal65, and Russ Fellows, Founder and VP of Labs at Signal65, are joined by Arm’s Jeff Defilippi, Senior Director of Product Management, and Ravi Malhotra, Director of Solutions Architecture to walk through Signal65’s analysis of Arm’s first production silicon, the AGI CPU, against AMD’s EPYC 9755 across seven CPU workloads, including how the study defines agentic CPU work and why the rack became the unit of performance.

For a hyperscaler, Malhotra frames the math as installed capacity: with 100 megawatts available, how many racks fit? And how many software development tasks can each complete per hour? 

The study models about 1.99 times the work per rack versus the EPYC 9755 on average, with workloads ranging from 1.43 times to 3.19 times, because each two-socket AGI CPU server draws 950 to 1,050 watts rated, against 1,700 watts for the EPYC 9755. The rack results are modeled from Arm’s rated server power on early systems, with follow-on testing planned on production hardware.

Key Takeaways Include:

🔹 Rack-level performance matters more than chip-level gains. Arm’s AGI CPU delivers greater compute density through lower rated server power (950–1,050 W vs. 1,700 W per two-socket server), translating a 1.2x average chip-level improvement into roughly 2x more work per rack.

🔹 Power is becoming the defining constraint for AI infrastructure. At hyperscale, capacity planning increasingly starts with megawatts, not processors. Performance, power consumption, and cooling must be optimized together to maximize work per rack.

🔹 Real-world agentic workloads are reshaping CPU benchmarks. By capturing inference traces from agent benchmarks like Terminal-Bench and SWE-bench, Arm can isolate CPU performance from GPU variability and measure how processors handle actual agentic workflows.

🔹 The barriers to Arm adoption are shrinking. Native support across containers, orchestration tools, and AI frameworks like LangChain, vLLM, and SGLang is making migration from x86 significantly easier than in previous years.

🔹 Agentic AI demands a new performance metric. The measure of CPU efficiency is shifting toward how many agentic tasks a system can complete within a given time while meeting service-level agreements (SLAs), with consistent per-core performance as systems scale.

With production samples shipping to OEM partners including Supermicro, Lenovo, and ASRock and a ramp expected toward year-end, infrastructure teams can begin qualifying rack-level CPU density against their own power budgets.

Watch the full video at sixfivemedia.com, and subscribe to our YouTube channel so you never miss an episode.

Read the full Signal65 analysis, Scaling Agentic AI with Arm AGI CPU, or download the whitepaper. Learn more about Arm at arm.com.

Disclaimer: Six Five Media is for information and entertainment purposes only. Over the course of this video, we may discuss companies that are publicly traded, and we may reference their equity share prices. Nothing discussed during this webcast should be considered investment advice or a recommendation to buy or sell any security. We are not investment advisors, and you should not rely on this content as financial advice. Six Five Media collaborates with technology companies and industry leaders to produce research-driven interviews and multimedia programming for enterprise technology audiences.

 Ryan Shrout:

Hey everybody, welcome to another Signal65 video insights. I'm Ryan Shrout, president at Signal65. And we are here today to talk about really an incredibly interesting topic about how CPUs are revitalized again in the world of AI, in the world of agents. And we've got some great guests to join me today and talk about it. First of all, I want to introduce Russ Fellows on our Signal65 team who helped lead a lot of the testing and analysis and kind of work that went into this report. Russ, thanks for joining me.

‍

Russ Fellows:

Hello, thanks for having me.

‍

 Ryan Shrout:

And we also are joined by two esteemed colleagues from Arm. We've got Jeff DeFilippi, Senior Director of Product Management, and Ravi Malhotra, Director of Solutions Architecture at Arm. Thank you both for joining us today. Thanks, Ryan. Nice to be here. So what we're talking about is actually maybe a little bit different for conversations we've had with or about Arm in the recent past, right? This is the Arm AGI CPU. This is really interesting, right? It is not architecture, it's not IP. This is actual silicon that is gonna be deployed and utilized by enterprises and hyperscalers around the world as a new competitor in the CPU space. I want to set up the stage a little bit for what this report is and kind of what we analyzed and what we looked at, right? This is looking at the ARM AGI CPU versus an x86 competitor, particular the AMD EPYC 9755, one of the other leading processors kind of in the server space that was set up as the competitive comparison here. We really wanted to look at this to try to analyze where the ARM AGI CPU sits in the landscape of data center CPUs out into the world. And this is a product that was launched earlier in the year that is now starting to come into production, starting to be released out into the wild. And we're really excited to get our hands on it even further down the road here later this year. We looked at, Russ, seven different CPU workloads. across three agentic categories, right? We've looked at tooling, testing, agentic workflows, reasoning and planning. I'm curious, like, what do you think about the kind of the collection of tests that we wanted to evaluate here and what they represent, right? Like, is it fair to say that agentic CPU work has kind of become this all-encompassing thing that we have to evaluate and talk about now?

‍

Russ Fellows:

It is and it has become, and interestingly enough, tests are changing incredibly rapidly daily. I'm working on new tests all the time to try and keep up with the rate of agentic changes, right? So as the agents change and become more capable and the tool calling expands, becomes a bigger part of the workloads, we have to change the test harness to keep up with that. So yeah, this is quite new and quite interesting and very important.

‍

 Ryan Shrout:

And some of those tests that we looked at, you know, were MySQL, Nginx, Terminal Bench, SuiBench. We also looked at, interestingly, kind of separate a little bit, SandBench, which is like a sandbox creation test. So all super important things to evaluate. Even more interestingly, or part of the story at least, is that we were really looking at this at a per rack level right the the kind of the primary claim that we took out of this is it's a 2x the work per rack right so this is the arm AGI CPU, you know 1.99 2x the modeled performance per rack versus that epic competitor. Do we think, I think we're used to or we're used to seeing discussions about the performance of the chip, right? And this study kind of argues that the rack is the right unit of performance. And I'm curious kind of what your thoughts are there on the power and space constraints and kind of how it's kind of reorganizing or refocusing the data center world.

‍

Russ Fellows:

Yeah, that's a hard one because you know what unit of measure do you pick and everybody will probably have some different answer that they like the best but at the hyperscaler and above the Neo clouds, the large data center build outs, the rack level probably is the right level of discussion because it's not about how many cores I have, it's about how many megawatts of racks do I have, right? So it's numbers of racks and it's all about power and the trade-off between those. Power performance, cooling size, all those come into play.

‍

 Ryan Shrout:

This is a good time to bring in our guests that have some input on this. Jeff, I'm curious your thought as well, right? As we set out to kind of do some of this work and do these evaluations and kind of how ARM was looking at talking about launching and making claims about the AGI CPU, what was it about those rack level numbers that made it make the most sense to you? Is it really the conversations you're having with these potential customers today?

‍

Jeff Defilippi:

Yes, absolutely. When we go into a customer, the first place we start is what rack are we trying to fit into? What's your power budget? And then it becomes a game of how much compute capacity can you squeeze into that power budget. And it was really a fundamental design point for the AGI CPU when we went off and designed it. We designed it for that 300 to 400 watt air-cooled system. And to be able to fit in a 1U, which is the height of a chassis, we were trying to fit two of those CPUs inside that form factor. So we were trying to maximize our compute density. But the starting point was that RAC power budget.

‍

 Ryan Shrout:

I'm curious, I kind of alluded to this at the beginning of the conversation where this is your first production silicon, right? This is a pretty significant shift for a company that I've known for a very long time that probably everybody watching has known for a very long time as an IP licensing and IP design company. Can you describe briefly kind of what is the AGI CPU and how do you envision it kind of fitting into the ecosystem as it sits now?

‍

Jeff Defilippi:

Yeah, sure. I mean arm started on this infrastructure journey. You know when I when I joined arm over over a decade ago we were on that track and we announced our universe product line back in 2018 with with AWS is the foundational customer. And obviously, we've seen a growth. We now have 1.5 billion cores deployed in the cloud right now. So we build on that foundation. But to take our IP, to develop a piece of silicon, produce a production quality piece of silicon, takes a large investment. And certainly, we have a lot of customers that want to take IP. They want to customize their solution from the ground up and customize silicon. But there was a lot of other customers that just wanted to buy ARM, and there was a gap in the market. And because we were building up to the foundation of building 64, 96, 128 core systems inside ARM, we were at the point where we felt comfortable that we could actually produce a product of the quality of AGI CPU.

‍

 Ryan Shrout:

Now, a couple of the interesting numbers from the report that we've posted on Signal65.com is that the AGI CPU is somewhere like 1.2x the performance of the competing EPYC processor that we compared it to. But at the rack level, that grows to the 2x number that we talked about before, just because of the number of servers you can fit into that rack space. Are there particular design choices that ARM utilized to try to deliver that performance in a kind of a compressed power scale relative to what some of the competing parts are doing?

‍

Jeff Defilippi:

Yeah, certainly if you think about, again, we targeted that 300 watt point so we could fit in that 1U space. And because we have such an efficient core design, we were able to maximize the compute density. And again, when you think about building these types of servers, you're thinking about balancing. You're thinking about a high-performance core with the right amount of cache, the right amount of memory capacity and bandwidth per core. and to tie that out with the networking system. So when we took those design constraints, because we had such an efficient core design, we're able to pack more at the rack level.

‍

 Ryan Shrout:

It was a very interesting kind of set of math and results to drive through to kind of get us to where those rack level numbers are. So I'm very curious to see how that continues to shift and change in the market as the ARM AGI CPU comes through. Ravi, I have a question for you like part a lot of the part another big part of this project really was kind of defining what even a gentic CPU performance meant. Right, it's a very new problem it's I think it's something that you know as Russ talked to at the beginning is. still kind of softly defined. And it really hasn't been rigidly set in stone yet. And I think that's part of what makes all this super exciting and super interesting. But how did you and Arm really think through what workloads represent agentic CPU work? And how does that, you know, decision, you know, kind of influence where we ended up in the white paper with these different component level and workflow level tests?

‍

Ravi Malhotra:

So when we started looking at Genentech AI, we started with looking at the different use cases, the different types of applications that are going to run this new era. And we kind of broken it down into five different categories. And all of these agentic applications, they behave differently from your traditional applications because it's a pipeline of operations that spread between CPUs and between your GPUs. And if you compare any two different pipelines or trajectories, no two of them are the same. And then we also looked at a lot of the new agentic benchmarks that are being developed, such as Terminal Bench, Sway Bench, Tau Bench. And a lot of these benchmarks we found are actually being developed by the same companies for building real world applications. There are a lot of universities like Stanford, for example, who are developing these benchmarks. And while originally these benchmarks were intended to be model quality benchmarks. We found a way to freeze the trajectory, capture the inference traces, and then turn them into CPU benchmarks. So each one of these agentic benchmarks is what I believe a good proxy of a particular type of application, such as SwayBench and TerminalBench are good proxies for software development. Now, while these new agentic benchmarks give you a good idea of what the agentic workflow looks like, often when you want to narrow down on a particular part of the pipeline, you have to pick a specific component. And that approach is also not new. We have done this in the cloud world as well. picked up databases, web servers, and quantified their performance. For agentic AI, we took the same approach, but then we expanded the scope of the components we picked to include things like vector databases, model serving, sandbox orchestration, stuff that is new that you would need in an agentic application.

‍

 Ryan Shrout:

Yeah, it's interesting. And Russ, I want your view on this too, because I think everything Robbie's walked through there is very true. But there's also like, there's a level of discussion around like, well, if an agent can go do anything, then it really, you need to benchmark everything, right? Anything that could possibly happen on the CPU. And it's kind of like a resurgence of kind of general purpose compute as one of the metrics of value as we go look through the kind of the AI infrastructure landscape.

‍

Russ Fellows:

Right, I agree. It is very much general purpose because, yeah, it's impractical to try and test everything. And Ravi's point was good in that we tried as best we could to take out the GPU variability because there's a lot of difference if you're running on an A100 versus an MI400X or a B300 node, the GPU power changes significantly. So how do you fairly compare different, you know, agentic pieces on the CPU? So the trace methodology works well to do that, to take out that variability and make the testing at least as fair as we can make it.

‍

 Ryan Shrout:

Yep. Robbie another one of the things that I thought was interesting you mentioned databases web servers vector search and all of that. Obviously all the testing that we were doing was running on arm 64 containers right these all ran natively on arm, and that has been an area that has been. maybe not slowly, but progressively getting much better. You know, what does the software path in your mind look like for a customer moving agentic workloads from traditional x86 servers to something like Arm AGI?

‍

Ravi Malhotra:

So the ARM software ecosystem has come a long way since, you know, 2018, 2019, when Neoverse was launched, thanks to folks like AWS, GCP, Microsoft Azure, launching their ARM compute instances. A lot of the software development work, it has started to be ARM native from the start itself. And a lot of the pieces that you typically need to build an agentic application, whether it be what containers or micro VMs you're running on, what orchestration framework you're using, whether it's Lang chain, what model saving from framework using PLLM, SG Lang, a lot of those bits and pieces that you would need for agentic application, They already support ARM, and most of the new ones that are being developed today, they are being developed with this multi-Arc, both ARM and x86 compatibility in mind. So anybody who's building an agentic application today, they would start by looking at what are the components they need, and they'll find most of them are already running on ARM today. So the whole porting effort or migration effort is not kind of a big deal as it used to be like eight or 10 years back. And by the way, now you have AI tools, ARM has MCP server. So pick codecs, pick cloud code and something you can use AI for that.

‍

 Ryan Shrout:

Yeah, yeah, it's, it's, it's, it's impacting everything that we're doing as well. I'm curious like if you look at sweet bench terminal bench. Obviously we'll have signal 65 pinnacle at some point as well that we're gonna we're gonna measure on the AGI CPU but if you Look at the claim of roughly 2x the agent work per rack. Can you describe what it means in like a practical term for a hyperscaler or an AI cloud provider that is trying to scale these coding agents or tool heavy workloads in their data centers this year, next year, 2028?

‍

Ravi Malhotra:

OK, so a hyperscaler would typically look at it from a installed capacity perspective. So if they've got 100 megawatts of installed capacity, how much of useful software development work can they sell to their customers? So how many racks can they fit within the capacity? And out of those racks, how many software development tasks can be completed per hour. So and again, no two tasks are of the same type. You know, Sway Bench, Terminal Bench, they have a set of representative tasks. But we take them as proxies to say, hey, you can complete more tasks in the same rack power budget. the same install capacity versus existing systems today.

‍

 Ryan Shrout:

Yeah. And Jeff, I'm curious from your point of view, you know, the, the signal 65 report that we published, you know, frames, this is about, you know, half the racks for the same work across, you know, a bunch of different workloads. Does that translate differently for like colo customers on prem versus cloud? I think, I think you guys are focusing really on cloud, but as we think about large enterprise, maybe, is there still a space for them to kind of evaluate that, that price performance advantage?

‍

Jeff Defilippi:

Yeah, yeah, certainly. I mean, if you think about it, certainly hyperscalers is one of our target markets. But, you know, there's new clouds popping up all over the place, you're going to have, you know, data centers and places where you're still going to be constrained by power and you're still going to see the benefits of having more compute density per watt. Everyone's thinking in terms of how many racks can I fit? What's my power to that data center? How many racks can I fit? And then how many compute nodes can I fit in that rack is how everyone's thinking about the problem today.

‍

 Ryan Shrout:

A couple of last things here, right, as we wrap this up. These results that we're looking at today that we're talking about, everybody should go to signal65.com and find this report and really take a look, right? They reflect early AGI CPU systems. You know, we have plans for follow-up testing on kind of final production hardware as well with our own tests as well as others. But I'm curious, like, what should, this is more of a forward-looking question than say what's happening next month. What should the market expect next for AGI CPU availability and kind of the roadmap for what ARM has planned. I know you're still getting that first generation now, but we're always curious, like what's kind of like the long-term vision?

‍

Jeff Defilippi:

Yeah, right now we have production samples that we're shipping to a number of our customers, and that includes OEM partners such as Supermicro, Lenovo, Asrock. So they're all getting their systems through the qualification phase now. So we're hitting that production ramp window here towards the end of this year. So you're going to start seeing a lot more availability of the product, and it's going to start showing up in a few more spots. You know, and then from just a benchmarking perspective, etc, you know, especially in the agentic world, we are starting with software development, we feel is a primary use case that has a lot of value to a lot of folks, but there's going to be enterprise, there's going to be robotics, there's going to be, you know, the personal assistant. So, you know, this is kind of, and we're working with our partners, and that includes, you know, both silicon partners, IP partners, you know, what's the right methodology, and what should we be focused on from from our ecosystem perspective for Gentic. Those are all things that are new and continue to be developed.

‍

 Ryan Shrout:

And I think maybe this last question I want to pose to both of you and get a viewpoint right for a hyperscaler or Neo cloud, or even like a large scale enterprise that is looking to plan and deploy agentic infrastructure in the near term. Like what is the, in your mind, the single strongest argument for putting the ARM AGI CPU at the top of that list, on the top of that short list?

‍

Jeff Defilippi:

I mean, certainly you got to start with the performance per rack. But I think when you think about the performance that we're bringing, we give a consistent ramp of performance. As you add cores, the balance of core to memory to IO, you're going to see consistent work performance per core. So the amount of agentic work in that environment where you're still getting that work done, but you're getting that work done on time. You're meeting your SLA requirements. That's the fundamental value that we're bringing.

‍

Ravi Malhotra:

Yeah, I'd like to frame this as it's not about how many agents can you run simultaneously. It's more about the balance of throughput and SLA. So how many agentic tasks can you get completed within a certain period of time, while still meeting a goal of an SLA at a task level?

‍

 Ryan Shrout:

Yeah, I think that's the right way to look at it. It's definitely a more complex way to evaluate performance, kind of looking at things in that light. But I think that's the task that Russ and myself have going forward as we continue to evaluate all these different AI infrastructure solutions. So, well, I wanna thank you both, Jeff, Ravi, for joining us, Russ as well. Really appreciate your time today to kind of walk through this.

‍

Jeff Defilippi:

Thank you.

‍

 Ryan Shrout:

For everybody watching, go to Signal65.com, look up this report. Russ did a great job kind of summarizing all the data and information and the claims as a part of it. We'll have a lot more discussions and evaluations on the ARM AGI CPU coming up shortly, I'm sure. For now, I'm Ryan Schraut with Signal65. Thanks for watching.

‍