I have;
Ts) TrueNAS box 1 with SFP+ PCIe card
Te) TrueNAS box 2 with 10GbE PCIe card
Sw) Mikrotik CRS305-1G-4S+IN 4-port SFP+ switch
Ts connected to Sw with a DAC cable
Te connected to Sw with an RJ45 transceiver at Sw
Testing iperf3 between Ts and Te from the TrueNAS shell I only get about 6.5 Gbps.
Is it worth replacing the PCIe card in Te with an SFP+ one? Or is my bottleneck between the two boxes likely something else? Or would 6.5 Gbps be normal for “10 gigabit networking”? This is all in the same rack, no long cables.
10GbE = 10 Gigabit Ethernet, regardless of what technology you use, be it 10GBASE-T or SFP+.
Test with iperf2, not 3, 2 uses multi cores, 3 doesn’t. SFP+ is always preferred over 10GBASE-T for using less power and getting less hot, and the added benefit that SFP+ supports everything.