I kinda want to put together a Ceph storage cluster — but I know it can take quite a bit to get good IOPS from ceph — good CPU’s and fast enterprise drives (but I want NVMe), oh and also good networking. But I mainly want to see what I can get in the IOPS department so sequential throughput, I’m not too worried about
How would you guys go about this? Any good hardware choices now that prices of things have come down a good bit in the last year or so?
WIth any kind of advanced software you will never be getting drive speed out of a soulution.
Raw nvmes can hit 800k iops. Add XFS and you may be able to get that.
with MDADM you get like 200-300k
ZFS shrink it to 20-50k
Anything network be glad if you get 5-20k
The more software is involved the worse performance gets especially for IO. Sequentials often scale better or even linearly, but IO is a PITA