Reduced performance on Sync server
Posted: 2018-01-23, 07:43
Just a notification to keep everyone in the loop on what's going on:
On Saturday, the SSD caching of the Sync server started failing due to Intel cutting corners in their storage solutions -- insufficient overprovisioning for wear leveling caused massive delays on writes, which resulted in the Sync server issues and errors for users last Saturday.
Our provider switched off SSD caching to mitigate and we're currently running on spinning disks only -- the node is stable but running at reduced performance levels as a result (caching is essential for heavy database use with tons of writes like Sync).
Replacement (larger) SSDs have been ordered for caching and will be put in (one at a time, to be sure) when they are on-site. We'll be informed when caching is switched back on.
On Saturday, the SSD caching of the Sync server started failing due to Intel cutting corners in their storage solutions -- insufficient overprovisioning for wear leveling caused massive delays on writes, which resulted in the Sync server issues and errors for users last Saturday.
Our provider switched off SSD caching to mitigate and we're currently running on spinning disks only -- the node is stable but running at reduced performance levels as a result (caching is essential for heavy database use with tons of writes like Sync).
Replacement (larger) SSDs have been ordered for caching and will be put in (one at a time, to be sure) when they are on-site. We'll be informed when caching is switched back on.