Article may be outdated

This article is 8 days old. Some details may have changed since publication.

Hacker News·5 min read·hard

Fitting Four 32 GB Resizable Bars Behind a PCIe Switch on Linux

D
dougcalobrisi
Fitting Four 32 GB Resizable Bars Behind a PCIe Switch on Linux
✦AI Summary

A hardware enthusiast details the technical process of configuring four 32 GB Resizable BARs on a Linux system using a PCIe switch. The article explains the memory mapping challenges and the custom kernel module developed to manage address space allocation for multiple high-end GPUs.

Why it matters

This demonstrates advanced hardware hacking and kernel-level optimization for high-performance computing setups that exceed standard consumer BIOS limitations.

✦Dive DeeperCreate a free account to unlock

My PCIe switch is a Broadcom PEX88096 in the board's one x16 slot, with four AMD Radeon AI PRO R9700s and the two RTX 3090s from my PCIe splitter post behind it, and the build post has the whole box.

The switch gives each card eight lanes and lets the cards copy to each other without going through the CPU. AMD's ROCm calls that peer-to-peer (P2P), and it only turns it on when each card's whole 32 GB is mapped. On Linux, that means a 32 GB Resizable BAR on every card. A base address register, or BAR, maps a range of the CPU's address space onto a device's memory. Resizable BAR is the BIOS option that lets a card's BAR be the full 32 GB instead of the default 256 MB.

Continue reading on Headlinne

Create a free account to read the full article.

Read full article →
technologyscience
✦

Get smarter about the news

Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.

Create free account

Already have an account? Sign in