Astera Labs Unveils Orion: A Compute-Agnostic Rack-Scale Architecture for Token-Efficient AI Factories

Modular, production-grade reference designs are designed to help neoclouds and enterprises quickly scale AI inference capacity up and across with up to 160 accelerators per pod

• Bring Your Own GPU/CPU: Modular Orion reference architecture is designed to give customers the flexibility to choose accelerators and CPUs based on availability, lead time, cost, target applications, or workloads.

• A faster-path to AI inference capacity: Six production-grade reference designs—spanning rack, compute tray, fabric tray, memory tray, link board, and compute board—help to accelerate time-to-market by reducing integration complexity and testing cycles.

• Fabric Efficient Token Economics: Orion’s high-radix 320-lane PCIe switch fabric is designed to scale from 8 to 160 GPUs per pod and to help the same GPU extract more tokens-per-second across AI inference workloads.

• An open, standards-based roadmap: Astera Labs plans to extend Orion’s roadmap to additional open interconnects, including UALink™, to help protect customers’ infrastructure investment and strengthen their supply chain resilience.

Astera Labs, Inc., a leader in semiconductor-based connectivity solutions for rack-scale AI infrastructure, announced Orion: a modular rack-scale reference design with production-grade connectivity that allows customers to choose their own GPU and CPU. To be offered through Astera Labs’ ODM and OEM partners, the Orion production-grade reference designs are expected to help enable neocloud and enterprise customers to deploy AI infrastructure faster and run it more efficiently, turning racks built on open standards into token-efficient AI factories.

For neoclouds and enterprises to economically serve inference at scale, they must secure sufficient accelerator and CPU capacity while delivering predictable, low-latency performance across increasingly complex AI infrastructure. But because they must commit to that capacity long before they know which accelerator will be available or affordable for their workloads, tying early investment to a single platform carries unnecessary cost and risk. At the same time, ODMs and OEMs are forced to repeatedly solve complex integration, validation, and software challenges for every new accelerator platform—slowing deployments when inference capacity is in critical demand.

Also Read: CIO Influence Interview with John Elliott, Cybersecurity Author Fellow at Pluralsight

Orion addresses these challenges with a production-grade reference architecture that decouples rack-scale designs from accelerator and CPU selection. Built on open standards to support industry-leading inference accelerators and CPU ecosystems, Orion is designed to enable operators to deploy infrastructure once and scale inference workloads across the most cost-effective and readily available compute resources—via PCIe plug-in cards—without redesigning racks as technology and market dynamics evolve. At the core of Orion is a high-performance, standards-based, scale-up fabric engineered to connect up to 160 accelerators per pod and deliver predictable, low-latency communication and expanded memory capacity across racks and pods. Orion’s optimized PCIe connectivity and composable infrastructure are additionally designed to help accelerators extract more tokens-per-second across AI inference workloads. Orion is delivered as a complete modular rack-scale reference design—including system configurations for the compute tray, fabric tray, memory tray, link board, and compute board—all integrated with COSMOS Rack Manager and COSMOS Fabric Manager orchestration software for management, fleet-wide telemetry, diagnostics, and AI infrastructure intelligence.

“We’re entering a new era of AI inference, where operators are judged on the tokens they deliver per dollar, not the hardware inside their racks,” said Pravin Patel, vice president and general manager of the AI Platform Solutions Group at Astera Labs. “Orion gives AI inference cloud providers a production-grade, compute-agnostic architecture so they can quickly bring inference capacity online, provide outcome-based pricing for their customers, and adapt as the industry and compute landscape evolves.”

Orion rack reference and system designs are on track for availability through Astera Labs’ OEM and ODM partners beginning in Q1 2027. Astera Labs will showcase a powered Orion rack—featuring interchangeable, pre-validated compute trays with different CPU and accelerator configurations at the 2026 Open Compute Project (OCP) Global Summit, taking place October 12-15 in San Jose, California.