Hello,
Earlier in the year I swapped out the mobo/CPU/RAM in my server:
- From: Core i5-4570S (Asus H87I-Plus), 16GB
- To: Xeon D-1520 (AsrockRack D1520D4I), 16GB ECC
The disks remain identical:
- 2TB WD Red parity
- 2TB + 1TB WD Reds data
- 2x 240GB OCZ SSD cache pool
They are now connected via a mini-SAS to 4x SATA cable, plus one directly in a SATA port on the board. All are reporting that they are connected at SATA 3.0 (6 Gbps).
I noticed today that, since this change, parity checks have been taking about three times as long! Below is my history - you can clearly see when I changed the hardware in April.
2019-10-01, 16:16:11 16 hr, 16 min, 9 sec 34.2 MB/s OK 0
2019-09-01, 17:44:43 17 hr, 44 min, 42 sec 31.3 MB/s OK 0
2019-08-01, 16:28:53 16 hr, 28 min, 52 sec 33.7 MB/s OK 0
2019-07-01, 16:24:50 16 hr, 24 min, 49 sec 33.9 MB/s OK 0
2019-06-01, 16:13:10 16 hr, 13 min, 9 sec 34.3 MB/s OK 0
2019-05-01, 16:40:44 16 hr, 40 min, 43 sec 33.3 MB/s OK 0
2019-04-01, 05:30:55 5 hr, 30 min, 54 sec 100.8 MB/s OK 0
2019-02-01, 05:30:07 5 hr, 30 min, 6 sec 101.0 MB/s OK 0
2019-01-01, 05:30:06 5 hr, 30 min, 5 sec 101.0 MB/s OK 0
2018-12-01, 05:30:06 5 hr, 30 min, 5 sec 101.0 MB/s OK 0
2018-11-01, 05:30:33 5 hr, 30 min, 32 sec 100.9 MB/s OK 0
I've tried to figure out whether it's just a parity check issue or a more general disk issue. I'm not a Linux guru, so I'm not sure if this is the best way to test this, but the outputs of the following dd commands seem to indicate I'm only getting 25-30 MB/s write speed on my data disks (and a much more reasonable 175 MB/s to my cache pool)?
root@Enthalpy:~# dd if=/dev/zero of=/mnt/disk1/test bs=1G count=20 oflag=dsync
20+0 records in
20+0 records out
21474836480 bytes (21 GB, 20 GiB) copied, 858.472 s, 25.0 MB/s
root@Enthalpy:~# dd if=/dev/zero of=/mnt/disk2/test bs=1G count=20 oflag=dsync
20+0 records in
20+0 records out
21474836480 bytes (21 GB, 20 GiB) copied, 715.322 s, 30.0 MB/s
root@Enthalpy:~# dd if=/dev/zero of=/mnt/cache/test bs=1G count=10 oflag=dsync
10+0 records in
10+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 61.4057 s, 175 MB/s
Based on some Googling I've tried disabling hot plugging and SATA Aggressive Link Power Management in the BIOS, but it's not made any difference.
Diagnostics are attached. I'd be grateful for any ideas, and am happy to try further diagnostics. Thanks!
enthalpy-diagnostics-20191101-1035.zip