Most "build your own inference cluster" guides stop at the GPU count and the model. The interconnect is usually one line: "use 400G". This post is about the parts that line hides, from someone who has spec'd and debugged these links.

1. What inference actually needs from the network

Training and inference have different traffic shapes. If you size the network for training, you will overspend; if you size it for a single-node demo, it will fall over at 8 nodes.

Training (data parallel):
AllReduce of gradients, MTU-sized messages, highly synchronized
Bandwidth-bound, tolerant of a few hundred microseconds of jitter

Inference, disaggregated (prefill and decode on separate pools):
KV cache transfer between prefill and decode nodes
Small control messages plus bursty cache blocks
Latency-sensitive: P99 tail matters more than average

The practical consequence: for inference, a single bad link shows up as periodic TPOT spikes, not as a throughput drop. Your monitoring has to be per-port, not aggregate.

2. The link budget nobody writes down

For a 400G or 800G scale-out fabric, the decision tree is roughly:

Under 3 m, same rack DAC copper
3 to 30 m, same row AEC (active electrical cable)
30 to 500 m 800G DR8 or 2xFR4, single-mode or multimode
500 m to 2 km FR4 / LR4

The mistake I see most often is using optics for intra-rack connections because "it's only a few hundred dollars more". At 64 ports per switch and 40 switches, that is not a few hundred dollars, and it is 20+ kW of extra heat in a room that was not designed for it.

3. Optics power is the real constraint

800G pluggable (with DSP) 15 to 16 W
800G LPO (linear drive) 8 to 10 W
CPO (co-packaged) 5 to 7 W equivalent

64 ports of 800G:
pluggable 1024 W
LPO 576 W
CPO 384 W

That is per switch. The difference between pluggable and LPO across a 40-switch fabric is around 18 kW, which is a different class of power distribution and cooling.

4. Why I still recommend pluggable for most builds

LPO saves power but costs you three things:

Interoperability: module and switch SerDes must be tuned as a pair. Vendor A's LPO module in vendor B's switch is often a support ticket.
Reach: typically limited to 500 m, and the link budget is tighter because there is no DSP to compensate.
Spares: you need same-vendor, same-batch spares. A mixed spare pool will fail in ways that are hard to diagnose.

CPO removes the DSP entirely but makes the optic non-serviceable. A single failed laser means an RMA of the whole switch. For a team without a dedicated optical lab, that is a bad trade at current maturity.

My rule of thumb: below 500 nodes, use pluggable. The power savings from LPO do not pay for the interoperability testing you will do.

5. What to actually measure

Acceptance test for a new fabric, per port:
1. DOM baseline: temperature, Tx power, Rx power, bias current, at install time
2. BER over 10 minutes with PRBS31, target better than 1e-12
3. FEC corrected/uncorrected counters, both should be flat at idle
4. Link flap test: 100 up/down cycles, no errors

Then ongoing:
1. Per-port CRC and FEC counters, sampled every 10 seconds
2. DOM trend, compared weekly against baseline
3. Temperature: anything sustained above 70 C goes on the replacement list

The single most useful signal is bias current trend. A laser that is aging shows a slow rise in bias for the same output power, months before it fails.

6. Cleaning, again

Half of the "bad module" cases I have investigated ended with a dirty connector end-face. A one-click cleaner costs less than a coffee. Make end-face inspection part of the install procedure, not part of the troubleshooting procedure.

7. A note on "just use copper"

DAC is genuinely better for short reach: lower latency, lower power, lower cost. The failure mode is mechanical: the cable is stiff, the bend radius is real, and a 3 m DAC routed badly will fail intermittently. If your rack layout requires tight bends, AEC is worth the extra cost.

-- F-tone Networks, Optical Interconnect Engineering
Technical resources: https://www.f-tone.com