Most of the AI infrastructure discussion I read here is about compute: more accelerators, bigger clusters, better kernels. From where I sit (I design optical links for a living), the binding constraint has been shifting for a while, and it shows up in very specific numbers rather than in headlines.

Where the bottleneck actually moves

1) Per-accelerator memory bandwidth grows slower than per-accelerator math throughput. That pushes workloads toward larger model-parallel and expert-parallel configurations, which are collective-heavy. All-reduce and all-to-all are not "compute plus a bit of communication" — at some cluster sizes they are the step time.

2) The interesting metric is not aggregate bandwidth on a datasheet, it is achieved bus bandwidth for the collectives your job actually runs, at your message sizes. A fabric can look fine on a bandwidth test and collapse on all-to-all with small messages because of tail latency and incast.

3) Energy per bit is the real limit. Electrical SerDes at 100G+ per lane has a reach problem: at 224G PAM4 per lane, copper is realistically a couple of meters inside a rack, and even that needs careful channel design and equalization. Anything beyond a rack has to be optics. That is not a preference, it is physics plus loss budget.

What that means in practice

- Scale-up domains (the NVLink-class stuff) are growing in radix. Once a domain exceeds what a single switch can hold, you either shrink the domain or introduce optics inside it. Both are legitimate; they have different failure modes.
- Optics inside scale-up brings new operational questions: laser reliability and burn-in, thermal density at the faceplate, and whether the transceivers are observable at all. A link you cannot monitor is a link you find out about during an incident.
- Linear-drive and co-packaged variants reduce the DSP/retimer energy per bit, at the cost of reach, interoperability, and serviceability. Co-packaged optics make the failure domain bigger: a broken laser is no longer a 20-minute swap if it is on the package.

What I would measure before buying anything

1) Step-time breakdown of your real training job: compute / communication / bubbles. If communication is over ~25-30% you are interconnect-bound regardless of what the marketing says.
2) Achieved all-reduce bandwidth at 3-4 message sizes you actually use, not just the biggest one.
3) Oversubscription ratio and the tail: p99 message latency under a realistic mix, not the mean.
4) Per-port energy per bit including the retimer, at the temperature your rack actually runs at. Datasheet numbers are 25 C numbers.
5) Failure and observability story: what happens when one lane in a 800G port degrades? Do you see per-lane pre-FEC counters, or only a port-down alarm?

A caveat on my own bias

I build optical links, mostly for industrial and long-reach analog transport rather than hyperscale DC (f-tone.com if anyone cares), so I am clearly not neutral on "optics everywhere". My honest view: copper wins inside the rack for as long as the loss budget allows, and optics should enter where reach or radix forces it — not because it is fashionable. The failure modes of a photonic link at the faceplate are worse than people expect, and the monitoring tooling is still immature.

Question for the crowd: for those running multi-node training on 100+ accelerators, what did your step-time breakdown look like before and after you changed the fabric? I would especially like to see the collective latency tails rather than the averages — that is where the real stories are.