A couple weeks ago I upgraded one my home lab machines I use for benchmarking. Aside from getting a new Ryzen, the main change was getting a bunch of Samsung 990 PRO M.2 SSDs for storage. And the performance I'm seeing is rather atrocious :-(
For most workloads (reads/writes, sequential/random) it works fine, e.g. for random writes I can get ~50k IOPS (iodepth=1) quite easily. But as soon as I ask fio to do fsyncs (fsync=1), the performance just absolutely tanks to ~200 IOPS.
@tomasv I've not found any Samsung consumer NVMe drives which benchmark well for me under direct / sync IO. The only recent consumer drive which was good was the Crucial P5, but they've dropped that and their later models are poor. It seems the current trend is getting PCIe 4/5 bandwidth and power loss protection is less important, so direct / fsync io has suffered.
@tomasv The best drives I've got which benchmark at 200k IOPs (4 jobs) are Samsung PM983s, there seems to be quire a few going cheap on ebay at the moment. Only slight issue is they are 22110 size, so mounting in most consumer motherboards is out the window.
@intrbiz The slightly annoying bit on PM983 is that they're rated only for 3GB/s reads, but I specifically bought the Asus Hyper M.2 to leverage the bandwidth limits, and with PM983 that won't be possible. I know, I'm asking for too much ...
@tomasv Yeh, probably because they're older Gen 3 drives. So limited to 32GT/s.
Maybe have a look for some Samsung PM9A3 to test. Or maybe some Micron 7450 Pro.
Depends how much you want to pay I guess, but either way you'll want to look for some write optimised enterprise drives.
This has been a handy list of benchmarks, which seems reasonably accurate (mileage may vary): https://docs.google.com/spreadsheets/d/1E9-eXjzsKboiCCX-0u0r5fAjjufLKayaut_FOPxYZjc/edit?gid=0#gid=0
