CPU inference is no longer a fallback for teams that cannot afford GPUs. For many regulated European workloads, running inference on bare-metal CPU nodes is an active architectural choice: deterministic latency, no multi-tenant risk, full control over the software stack, and a hardware cycle that can be planned years in advance. The question is which silicon to anchor that strategy on. AMD’s EPYC 9005 “Turin” and Intel’s Xeon 6 both landed in 2024, and both make aggressive claims. Neither is universally better, and the vendor benchmarks that fill most comparison posts are not independent. This guide cuts through the noise.

Before the numbers: every performance claim in this post that comes from AMD or Intel is vendor-sourced or vendor-commissioned. Independent, apples-to-apples benchmarks from a neutral third party are not yet widely available for this generation matchup. Treat the figures as directional signals for workload planning, not procurement commitments.

What each architecture actually brings to inference

AMD EPYC 9005 “Turin” (Zen 5, 5 nm TSMC) tops out at 192 cores and 384 threads per socket, with 12-channel DDR5-5600 memory support, PCIe 5.0, and CXL 2.0. The redesigned L1 data cache doubles bandwidth to both the cache and the FPU, which matters for token-streaming workloads that are memory-bound rather than compute-bound. Per-socket memory bandwidth sits around 540 GB/s. The chiplet design that enables these core counts does introduce NUMA complexity: workloads that cannot be cleanly partitioned across NUMA nodes may see higher latency than the raw bandwidth figure suggests.

Intel Xeon 6 splits into two distinct product lines. Granite Rapids (up to 128 P-cores, shipping September 2024) carries AMX (Advanced Matrix Extensions), Intel’s on-chip matrix engine purpose-built for INT8 and BF16 inference. Sierra Forest (up to 288 E-cores, announced June 2024) trades peak single-thread performance for density and performance per watt, targeting cloud-style task-parallel workloads. In February 2025 Intel extended the P-core line with the 6700P and 6500P series, adding more cores and double the memory bandwidth of the previous generation. Per-socket bandwidth is approximately 500 GB/s. For workloads that do not scale across many NUMA nodes, Xeon’s monolithic die design can deliver lower latency than EPYC’s chiplet topology.

The benchmark picture, such as it is

AMD claims its 192-core EPYC 9965 delivers up to 3.0 to 3.8 times faster AI inference across a range of workloads compared to Intel. On Llama 3.1-8B translation, AMD puts the EPYC 9965 at 1.33 times the throughput of the Xeon 6980P; on XGBoost it claims 1.93 times; on the TPCx-AI general AI benchmark, 1.7 times. For GPU-hosted inference, AMD’s 64-core EPYC 9575F (paired with eight Instinct MI300X accelerators) achieved roughly an 8% overall throughput increase over an equivalent Xeon Platinum 8592+ configuration on select Llama 3.1-70B workloads. The core count advantage is real: at 192 cores, the EPYC 9965 supports 33% more vCPUs than the highest-core-count Xeon 6 E-core part currently available.

Intel’s counter-claims centre on AMX efficiency. Intel states Xeon 6 delivers up to 1.5 times better AI inference performance than 5th Gen EPYC using one-third fewer cores, and projects up to 68% total cost of ownership savings through server consolidation. On MLPerf Inference v5.0 (the closest thing to a neutral benchmark for CPU inference), Xeon 6 P-cores showed an average 1.9 times improvement over 5th Gen Xeon in key benchmarks including ResNet50, RetinaNet, 3D-UNet, and GNN-RGAT. Critically, AMD does not appear in MLPerf CPU submissions. That absence is not evidence of inferior performance, but it means the only third-party inference benchmark data available covers Intel alone.

The platform that wins on a vendor slide deck is not necessarily the platform that wins on your workload. Until AMD submits to MLPerf, any comparison of raw inference throughput requires your own benchmarks on your own models.

Matching architecture to workload class

For large language model inference on CPU (serving smaller models like 7B or 8B parameter families without GPU acceleration), memory bandwidth is the dominant constraint. EPYC’s 540 GB/s per socket and higher core count give it a structural advantage for batched, throughput-oriented serving. If you are running vLLM or llama.cpp across a two-socket node, EPYC 9005 is the natural starting point.

For structured ML inference (gradient-boosted trees, XGBoost, LightGBM, classical recommendation models), the picture is more nuanced. AMD’s XGBoost claims look strong, but AMX on Xeon 6 P-cores accelerates quantised matrix operations in a way that benefits INT8-quantised neural networks specifically. If your pipeline mixes tree-based models with quantised neural networks, both platforms warrant testing.

For GPU-hosted inference with CPU as the host processor, the EPYC 9575F is purpose-built: 64 cores, high memory bandwidth, and PCIe 5.0 lanes to feed accelerators without bottlenecking. The 8% throughput gain AMD demonstrated over Xeon in this configuration is modest but consistent. The more important variable here is PCIe topology and NUMA affinity between CPU sockets and GPU slots, which requires hands-on validation with your specific accelerator layout.

For density-first deployments where power and rack space are the binding constraints, Intel’s Sierra Forest E-core parts are worth serious consideration. Up to 288 cores per socket at lower TDP per core suits environments running many small, independent inference processes rather than a few large ones. This is the scenario where Intel’s consolidation claims are most credible.

Energy efficiency: the metric that shapes the three-to-five-year cost model

CPU inference energy efficiency is not a single number. It is a function of workload type, batch size, quantisation level, memory access pattern, and cooling infrastructure. EPYC 9005’s top-end parts carry TDPs up to 500 W per socket; Xeon 6 Granite Rapids sits in a similar range at the high end. The efficiency story is in performance per watt at the workload level, not TDP alone.

AMD’s architectural improvements in Zen 5 (approximately 15 to 20% higher IPC than Zen 4) mean that for a given throughput target, you may reach it with fewer cores active, reducing effective power draw. Intel’s AMX argument is similar: doing more inference work per clock cycle on fewer cores, at lower total power. Both claims are plausible; neither is validated by independent data at the time of writing.

We provision bare-metal AI inference nodes at DIVERSITY across our AI infrastructure, and the pattern we see consistently is that energy efficiency projections made from vendor datasheets diverge significantly from measured consumption once real inference workloads run at production batch sizes. Budget for your own power measurement during evaluation, not after procurement.

Platform stability and the hardware cycle

A procurement decision for bare-metal AI workloads needs to hold up across three to five years. Two platform factors matter here beyond raw performance. First, memory and interconnect compatibility: EPYC 9005 is SP5-compatible, meaning existing SP5 infrastructure can be upgraded without a full platform change. Xeon 6 introduces a new socket (LGA7529 for Granite Rapids), so upgrades from prior Xeon generations require new motherboards. Second, software ecosystem maturity: AMX support in inference frameworks (PyTorch, ONNX Runtime, OpenVINO) is well-established; Zen 5 AVX-512 and VNNI optimisations are catching up but may require explicit tuning flags in some frameworks today.

CXL 2.0 support on EPYC 9005 opens a future path for memory pooling and disaggregation that could matter as inference model sizes continue to grow. It is an option, not a requirement today, but it is worth factoring into a five-year horizon.

The decision framework

  • High-throughput LLM inference, batched, memory-bound: start evaluation with EPYC 9005, specifically the 9965 or 9754 depending on core count requirements.
  • Quantised neural network inference (INT8/BF16), latency-sensitive: Xeon 6 P-core with AMX is the stronger starting hypothesis; validate with your own quantised models.
  • GPU-hosted inference, CPU as host: EPYC 9575F is purpose-designed; test PCIe topology carefully before committing.
  • High-density, power-constrained, task-parallel inference: Xeon 6 E-core (Sierra Forest) is the logical candidate.
  • Mixed workloads or uncertain roadmap: neither platform dominates; run both in parallel during evaluation and measure power at the rack level.

The honest answer to “EPYC or Xeon” is that it depends on which inference workload class you are actually running, at what batch size, with what quantisation, and under what power envelope. Both platforms are genuinely competitive in 2025 in ways they were not two generations ago. The teams that will make good procurement decisions are the ones that insist on running their own models on candidate hardware before signing a three-year contract.