Home

Astera Labs & Hyve on Orion: A Rack Built to Outlast the GPU

Astera Labs & Hyve on Orion: A Rack Built to Outlast the GPU

At OCP Global Summit 2026, Astera Labs and Hyve Solutions introduce Orion, an open PCIe rack-scale reference design that decouples the rack from the accelerator. For neoclouds and enterprises, it means scaling AI inference to 160 GPUs with any CPU or GPU while lowering cost per token.

AI infrastructure buyers face a widening gap between the GPUs they want and the platforms to deploy them. Every accelerator vendor brings its own rack and deployment model, compute is scarce and expensive, and cost per token keeps climbing as KV cache demands explode with agentic workloads. What if operators could upgrade their accelerators without rebuilding the infrastructure around them?

At OCP Global Summit 2026, Brendan Burke, Research Director, Semiconductors, Supply Chain & Emerging Tech at Futurum, is joined by Sanjay Gajendra, President, COO, and Co-Founder of Astera Labs, and Dr. Dimitris Ziakas, CTO and VP, Architecture at Hyve Solutions. They introduce Orion, a rack-scale reference design that puts up to 80 GPUs in a single PCIe domain per rack and 160 across racks, with any CPU or GPU.

The design separates the rack from the accelerator. Operators deploy connectivity, power, cooling, memory, and management once, then plug in whichever accelerators best fit their cost, supply, and performance needs as each generation arrives.

Key Takeaways:

🔷 Orion brings Astera Labs’ connectivity stack into one rack. Gajendra says Orion combines Aries retimers, Scorpio switches, Leo memory controllers, and COSMOS software into a single vertical stack. The goal is the highest inference performance at the lowest possible cost per token.

🔷 The rack is decoupled from the accelerator. Compute is scarce and every XPU vendor wants to sell its own rack, forcing operators to integrate a new one with each GPU change. Orion supplies the connectivity, power, cooling, and software so operators simply plug in an accelerator, while rack management stays the same across GPU upgrades.

🔷 Hyve turns the reference design into deployable racks. Ziakas says Hyve, part of TD SYNNEX, designs storage, compute, and fabric under one roof and uses global procurement to compose racks across vendors for each customer’s demand. He adds that KV caching over a large-radix PCIe fabric can unlock GPU cycles that are otherwise stranded.

🔷 High radix, in-network compute, and fabric-attached memory drive the gains. The 320-lane Scorpio X switch keeps switching to a single tier, in-network compute processes data in transit, and Leo memory sits on the same protocol plane as the GPU for KV cache. Gajendra cites 25% to 80% performance improvements depending on workload.

🔷 Open PCIe removes components from the rack. The same fabric handles scale-up and memory pooling, eliminating a dedicated CXL switch and scale-out NICs, along with their cost, power, and supply chain exposure. Ziakas says opening scale-up, long a closed ecosystem, lets operators compose heterogeneous compute and storage under a global address space.

🔷 Orion 1.0 is the first step on a roadmap. Gajendra says liquid-cooled versions, larger clusters reaching 320 ports, and support for multiple open scale-up protocols are coming. The targets are enterprise AI, inference-focused neoclouds, and new accelerator companies that need a ready rack for their silicon.

Operators that standardize on an open, accelerator-agnostic rack can add capacity as GPUs become available, without re-engineering their infrastructure for each new vendor or generation.

Watch the full video at sixfivemedia.com, and subscribe to our YouTube channel so you never miss an episode.

Learn more about Astera Labs at asteralabs.com and Hyve Solutions at hyvesolutions.com.

Disclaimer: Six Five Media is for information and entertainment purposes only. Over the course of this video, we may discuss companies that are publicly traded, and we may reference their equity share prices. Nothing discussed during this webcast should be considered investment advice or a recommendation to buy or sell any security. We are not investment advisors, and you should not rely on this content as financial advice. Six Five Media collaborates with technology companies and industry leaders to produce research-driven interviews and multimedia programming for enterprise technology audiences.

Transcript

Sanjay Gajendra:

What we are trying to solve is build a rack that is GPU agnostic and at the same time the management of the rack is the same although you are upgrading the GPUs along the way.

‍

Brendan Burke: 

Hello and welcome to Six Five On The Road at OCP Global Summit 2026. I'm Brendan Burke, Research Director at Futurum, and today we're looking at Orion, a new rack scale design for AI inference built on PCIe scale-up. Today, I'm joined by Sanjay Gajendra, President and COO at Astera Labs, and Dr. Dimitri Ziakas, CTO at Hive Solutions. Sanjay and Dimitri, welcome to the 6.5, my friends.

‍

Sanjay Gajendra: 

Great, thank you for having us. Pleasure to be here.

‍

Brendan Burke: 

Sanjay, super exciting week hearing about the Orion RackScale design. Astera Labs, I've been following you since you're a private company. You built your name as the leader in PCIe retimers with Ares, which we have in front of us here, then expanded into switching with Scorpio, CXL with Leo, and now you've evolved to a RackScale design. Back up and tell us about Astera Labs journey and how this progression from re-timers through to memory has now led to Orion.

‍

Sanjay Gajendra: 

Great. Thank you. Thank you so much. So we started Astera with a simple mission to build a connectivity stack that's purpose-built for AI infrastructure and AI workloads. So the vision was to build all the major components with its physical layer products like Ares, like you noted, memory controllers like Leo, fabric and switch products that we have at Scorpio. But the idea is that we were pursuing with Orion is to say, how do you combine all of this memory connected networking and data related components into one vertical stack so that we can deliver the highest level of performance as it relates to inference use cases or other workloads where tokens per second the money that you pay per token is as low as possible. And that's what Orion is trying to do.

‍

Brendan Burke: 

It's a fantastic moment to do it because we're now reaching these benchmarks and trying to push them forward. And so now thinking from first principles of how to build a rack is really what it's going to take to get us to the next level of performance. And Dmitry, I'm curious for your perspective on that because Hive Solutions, you're a major player in the market, part of TD Synex, and now the first partner to build the Orion rack. What about your history at Hive prepares you to take on a challenge like a RAC purpose built for inference?

‍

Dimitris Ziakas, PhD: 

Absolutely, and we're really excited to partner with Astera Labs actually on this. Hive, as you said very well, is all about RAC-level solutions. And that's where AI is. It's about RAC-level systems. And Hive is prepared for two major reasons for that. First is, that's what we do every day. We design and ship RAC-scale systems for AI. And Orion is all about composing that optimized rack-scale solution. So we build and design under one roof the storage systems, the compute systems, and the fabric from Astera to optimize for the best rack-level AI solution. And we have the TD Cinex arm, which gives us the global procurement scale to achieve that across multiple vendors, which is the whole point of bringing an open type of fabric and composition, because you can combine the best of all the different compute and storage vendors to achieve that.

‍

Brendan Burke: 

A lot of different components coming together. We have a few of them in front of us here, ranging from connectivity through to memory pooling. And at Futurum, when we talk to hyperscalers and neoclouds, the problems they're running into when scaling up their token output isn't necessarily what the spec sheet is for a GPU, but really it's the interconnect as well as the memory bandwidth that they experience when distributing workloads across systems in their data center. And now as inference scales up as well as agents, KV cache is exploding in terms of the terabytes that it can take for even an individual user to be able to do a long running session. And Sanjay, from Astera's perspective, which challenge did you design around and hear from customers that they needed a new solution for? Because it sounds like Orion is really meant to lift the ceiling of experience throughput in the data center.

‍

Sanjay Gajendra: 

Yeah, absolutely. See, the fundamental thing is everyone needs compute today. It's the most sought after thing in the world, but at the same time, it's also not available. If you want to buy some of these compute chips, it takes multiple weeks to get the chip, even if you can get one, and it's also very expensive. So the foundation problem that we're trying to solve is how do you build a rack that includes all of the connectivity that's needed, but it's GPU or CPU agnostic? In other words, a rack brings in the connectivity, the power, the cooling, the software. The only thing you need to do is plug in the accelerator that best serves your commercial requirement, cost, supply chain, and so on, as well as the performance specs that you need. In other words, the rack is meant to be completely agnostic to what the XPU is being put in there. That's the first one. The second one I would say is that nowadays, every XPU vendor wants to give you or sell you a rack. If you are a data center operator, especially for Neo clouds or enterprise AI deployment scenarios, you are now tasked to actually integrate a new rack every time there is a new GPU from a new vendor and new version and so on and so forth. So what we are trying to solve is build a rack that is GPU agnostic, and at the same time, the learning, the management of the rack is the same, although you're upgrading the GPUs along the way. So those are some objectives that we've been able to achieve, and like noted in my initial point, By integrating memory, networking, and data movement type of chips in a very tight fashion, including our Cosmos software that does now the orchestration of all the GPUs and fabric management, rack management, we're able to provide a solution that not only achieves a really high performance for various different workloads, but it's also scalable, it addresses the foundations of the challenges as it relates to availability of GPU and the portability or optionality across multiple GPU vendors.

‍

Brendan Burke: 

It's been amazing to see how Astera has been able to drive distributed computing in terms of sending jobs across GPU clusters, and to think about how that will play out in a REC scale solution where scalability has been the challenge for some more emerging XPUs. And we've tried some legacy network protocols. They didn't work very well, and then we've had to upgrade them. So this will be an incredible capability for new accelerators on the market. And so, Dimitri, you work across different types of systems and different types of compute, and now this is becoming a reference design that you can implement for your customers. When you hear about connecting data movement to GPUs in a more streamlined fashion, how are you going to take this reference design and then turn it into a finished product that your customers can use?

‍

Dimitris Ziakas, PhD: 

Absolutely. So the pre-validated and really well-engineered product from Astera comes in and provides that high-rate XPCIe fabric that allows us, as we discussed, to choose multiple different XPU iterations for our compute, configured with different storage configurations, and really bring the rack-level solution to our customers. So what we do is we then look at specific customer demand, how we optimize And we take advantage of that great capability to compose different rack designs for our customers and bring them at scale and really fast time to market. As Sanjay was pointing out, one of the core perhaps difficulties in the market right now is availability. of parts and actually sometimes also unlocking those cycles in those parts. So you mentioned KV caching, KV caching through a PCIe large radix fabric can unlock GPU cycles. So in your infrastructure, composing that at the rack level can really optimize how you deploy for the tokens and how you optimize. That's what Hive does best. And that's how we partner very well to provide those rack level solutions to customers.

‍

Brendan Burke: 

It's great to hear you can benefit from the Scorpio switch, and I do want to address what we have on the table here, because we have our switchboard with eight slots for GPUs, but then we also have the Leo CXL appliance here. imagination of Orion is that you could put a CXL board directly in the rack in what has been a discrete server or a software-based implementation attached to a CPU. Sanjay, I'm interested if you see CXL as benefiting from this high throughput connectivity in a way that can accelerate how memory transfers within the rack.

‍

Sanjay Gajendra: 

Yeah, absolutely. So let me explain that by pointing to this board here. So this one, the green board here, is what we call a compute board. It has our Scorpio fabric switch on that and eight slots where you can plug in GPU cards, you can plug in the Leo memory cards, or any other PCIe-based device, storage devices, and so on. So what it does is it allows you to create a platform that's very tightly connected with each other, meaning the GPU, the memory, along with any other storage and other components. And you're essentially able to connect all of that with the high-rated switch that we have in order to keep the switching at a single tier. So the latency is lower because of what we call fabric-attached memory enabled through our Leo product line. You are able to now access a ton of memory for KVCache applications, which significantly improves the inference performance across a variety of metrics. Depending on the workload, we've been able to demonstrate that by leveraging the Orion system, you can go anywhere from 25 to 80 percent of performance improvement through this architecture. So it's meaningfully solving the challenges that the industry is facing in a modular fashion, in an open standard way, and in a deployment scenario that allows someone to scale from eight GPUs on a single a 4U server box to using our scale-up technology to connect up to 10 of these boxes in a single rack to have a port that goes to 80 accelerators. So in general, it's meant to be very scalable as well.

‍

Brendan Burke: 

That's going to be a really unique form factor for customers to get to know because we're kind of at a 72 GPU scale-up domain commonly in the market due to connectivity limitations that might progress towards 144 and beyond in the future. But 80 is already a step up in terms of the number of GPUs to connect as we get these systems out to market. And then within a scale-up domain, you can get to 160 as well, just based on the number of lanes available on Scorpio. And so, considering the fact that you're incorporating, you know, the memory pooling and, you know, this extensive scale-up domain, how do you accomplish that? And is that where those gains come from of 20 to 80% throughput?

‍

Sanjay Gajendra: 

So there are probably three different reasons for achieving that kind of performance. One is the large radix of these fabric devices. Each device that's shown here goes up to 320 lanes, so you can imagine future versions will support higher. So it starts with the Radix because you are able to now connect a large number of accelerators in the same switching plane. So you don't have to do like a multi-tiered switch, which if you use like legacy product, you end up doing the same thing. Like if you use something that's got 144 lanes, you're now essentially forced to add multi-level switching in order to create a bigger Radix size or port size. You don't need that with these devices. So that's number one. Number two, these devices not only are doing the switching function, but they also have advanced capabilities like in-network compute, where there is computation happening in the data as the data is traversing through the chip. And that eliminates additional cycles that would be needed from a processor, whether it's a GPU or a CPU. Again, advantage is lower power, fewer number of components that may be required while getting meaningful upgrade in performance. The third reason why we are able to achieve that is by having fabric-attached memory where the Leo product can talk CXL protocol, it can talk PCI Express protocol, it's directly sitting on the same exact protocol plane where you have GPU. So it's almost like the memory is readily available and there's no translation happening from a protocol standpoint or any other additional overhead, and you're able to get the most optimized performance through that. So to summarize the combination of high radix, the features that we built in the Fabric device, and the integration we have done with the memory devices and other devices that we have brought together by the Cosmos software, we're able to achieve some of the performance numbers that I noted.

‍

Brendan Burke: 

Well, super critical to eliminate networking tiers when we can. I mean, we're talking about scale up across racks, but that only introduces hops and potential bottlenecks along that process. And so being able to have a compute domain of 160, I think is going to enable some much larger models and push us towards multi-trillion parameter models in a scale up domain. Dimitri, when you hear this, how do you think the market is going to receive this breakthrough and scale-up capability?

‍

Dimitris Ziakas, PhD: 

Scale-up is a critical domain that was known for being a closed ecosystem that was engineered specifically between similar CPUs or GPUs. So opening that up to the market really unlocks how you can compose a rack solution, to be honest. So I'm really excited to see this coming in from Astera. I'm really excited as to what it unlocks in the market. the different compositions that we mentioned earlier, the ability to really optimize across different heterogeneous compute and storage components, that really unlocks a key capability for us. And with a memory fabric at the back, the global address that you can create for all these components really can optimize how your workload runs on it. So it's a really exciting moment for sure.

‍

Brendan Burke: 

I think it's a key part of OCP, the idea of interoperability and the idea of purpose-built XPUs as well as agentic workers for different tasks. Sanjay, where do you see this going? What do you think customers will be able to imagine with this open fabric?

‍

Sanjay Gajendra: 

Yeah, I think before I answer that question, let me just add on to a couple more things that Dimitri said. You know, I think open standard is super important, because this is PCI Express. It's been around for forever. Great ecosystem when it comes to IP, testing, all the test equipment that's needed, and so on. And we are able to leverage that to achieve performances that are really remarkable. The second thing I would add to this is, the nice thing about this architecture also is that you don't need several extra components. You don't need a CXL switch to do pooling. The same fabric that's used for scale-up is also able to support the memory devices. One extra component means one extra component you need to pay for power, and of course, given the supply chain, it's one less thing to worry about. You don't need to scale out NICs and all this stuff. We can go up to 160 GPUs in two ports. So again, extra components that you would generally have to pay for now, you no longer require that with the architecture that we have put together right now. Now, in terms of where are we going with this, we like to say we're just getting started. Now that we have been able to build an integrated solution that is truly solving some foundational issues in terms of compute availability, the costing of the systems, power consumption, as well as scaling that's required and the modularity that's required. Now, it's about how do you make this address various kinds of workloads and cost points. because we are targeting the enterprise AI applications, we're targeting inference, new clouds that are popping up, we're targeting new accelerator companies that are coming up with their own unique and very interesting accelerator architecture. They don't have to now worry about a rack cluster that meets their requirement. This is based on what we call Orion 1.0, that's based on PCIe. We'll have versions of this that's not just air-cooled, but liquid-cooled, for example. We will support larger cluster sizes, meaning going from 80 to 160 to more, like 320 and so on, in terms of port sizes. We'll support multiple scalar protocols, but all open standards. We're a big believer that AI has to be enabled to all spectrums of the market, not just like a domain of few companies or a few countries for that matter. So what we're trying to do is use open standards, provide a modular architecture so you can scale it to how many ever GPUs that you want from a cluster size standpoint, different price points and performance points that you want to achieve. So stay tuned. This is just the first of the many architectures that we'll be putting out.

‍

Brendan Burke: 

There's going to be a lot to follow, but thank you for eliminating the CXL switch. That's one less product line for analysts to track. And there are some products here that I'm very familiar with already. So looking forward to seeing how these come together in a single solution. Thank you, Sanjay. Thank you, Dimitri, for sharing your innovation with us. To the audience, thank you for joining us for Six Five on the Road from OCP Global Summit 2026. Be sure to follow us on socials, subscribe, and follow all of our OCP coverage on sixfivemedia.com.

MORE VIDEOS

AI Ambition Meets Infrastructure Reality: HPE and AMD, Silicon to System

HPE’s Krista Satterthwaite and AMD’s Dan McNamara join Patrick Moorhead and Daniel Newman to explain how agentic AI is multiplying CPU demand and why silicon and system design now have to be tuned together for threads per watt. For enterprises, matching each workload to the right platform and model determines whether AI value outpaces the cost of running it.

Jensen Huang Steps Down: What NVIDIA’s Leadership Shock Means for AI, Washington, and the Market

Few weeks in tech have moved this fast. Within days of the first reports of an alleged affair, Jensen Huang resigned as CEO of NVIDIA, the company at the center of the AI buildout, leaving Wall Street, Washington, and the entire AI supply chain asking what comes next. Patrick Moorhead and Daniel Newman devote most of the episode to the NVIDIA story: the leadership transition, the unresolved questions, the export control fight it reignited, and whether the company needs an operator or another visionary. They close with Bulls and Bears on Anthropic's IPO filing and earnings from Applied Materials, CoreWeave, and Cisco.

How a Research Institute Got a 100x Speedup Moving Biology Sims to TPUs

At the Google Public Sector Summit, Caltech’s Matt Thomson explains how virtual tissue models running on Google Cloud TPUs are turning biology into a compute problem, with a 100-fold speedup in therapeutic simulations. Pairing that infrastructure with clean data could compress the path from research insight to new therapies.

See more

Other Categories

CYBERSECURITY

QUANTUM