磁盘延迟、并出现extcache.io.writeError和insert_Reject限制_io
适用场景
- ONTAP 9
- 所有采用基于NVMe的闪存的FAS 系统
问题描述
Disk
延迟中心延迟增加
cluster01::> qos statistics volume latency show -vserver vserver01 -volume volume01
Workload ID Latency Network Cluster Data Disk QoS Max QoS Min NVRAM Cloud FlexCache SM Sync VA
--------------- ------ ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ----------
-total- - 24.06ms 104.00us 53.00us 556.00us 23.35ms 0ms 0ms 3.00us 0ms 0ms 0ms 0ms
volume01-wid51230 51230 22.49ms 104.00us 52.00us 526.00us 21.81ms 0ms 0ms 3.00us 0ms 0ms 0ms 0ms
-total- - 13.60ms 143.00us 60.00us 188.00us 13.21ms 0ms 0ms 2.00us 0ms 0ms 0ms 0ms
volume01-wid51230 51230 13.19ms 143.00us 59.00us 185.00us 12.80ms 0ms 0ms 2.00us 0ms 0ms 0ms 0ms
- 多个事件
wafl_exempt09: ems.engine.suppressed:debug]: Event 'extCache.io.writeError' suppressed 67626 times in last 601 seconds.
wafl_exempt09: extCache.io.writeError:notice]: WAFL external cache I/O write error: Insertq_save: unable to set up context chain, code 0
- FlashCache不再插入
cluster01::> node run -node node02 stats show -p flexscale-access
Cache Reads Writes Disk Reads
Usage Hit Meta Miss Hit Evict Inval Insert Chain Blocks Chain Blocks Replaced
% /s /s /s % /s /s /s /s /s /s /s /s
45 703 11 39171 1 0 0 0 0 0 0 0 109
45 320 26 37607 0 0 0 0 0 0 0 0 98
45 903 51 38018 2 0 0 0 0 0 0 0 114
- 重新启动会暂时修复此问题