由于达到 FCP 队列深度阈值而产生的性能影响
适用于
- ONTAP 9
- 光纤通道 (FCP)
问题
- 生产中断可能以极高延迟的形式发生,类似于性能问题
- 适配器重置在达到队列阈值后不久
- 重复的 STIO TPD cmd alloc 失败消息表明队列已满
- 例如,在基于客户端的备份作业期间,从 qos statistics volume latency show 中观察到网络子系统出现高延迟
EMS 日志示例:
fcp.io.status:debug]: STIO Adapter:0g LUN:13DROPPED Cmd[2A] from SID:240942 VP:2 on OXID:000766 RxEx 0xffffffff (cmd allocation exceeded)at 59908347756
Tue Jul 16 11:30:56 +0700 [Node01: fct_tpd_work_thread_0: fcp.io.status:debug]: STIO Adapter:0e AEN 0x8048 (RECV_ERROR) MboxStatus1 0x1200 MboxStatus2 0x44
fct_tpd_thread_3: fcp.io.status:debug]: STIO TPD cmd alloc threshold reached handle:3 taskflags:0 Active commands:1945 threshold:1945
fct_tpd_work_thread_0: scsitarget.fct.port.thresh:notice]: FC target port 2d has 1946 outstanding commands, which exceeds the threshold 1945 for this port.
fcp.io.status:debug]: STIO Adapter:3a, found hung cmd:0xfffff81e7a302e40(state=7, flags=0x0, ctio_sent=1/2,RecvExAddr=0x1c4, OX_ID=0x462f, RX_ID=0xffff,SID=0x5c0f14, Cmd[2A], req_q_free:0)
fct_tpd_thread_4: scsitarget.fcp.dump:debug]: FCP target SRAM dump generated for adapter 3a, fct_tpd_check_hung_commands: Command termination hung. cmd:0xfffff81e7a3534f8 (state=0xa, flags=0x2,ctio_sent=2/2, RecvExAddr=0xa7b, OX_ID=0x16b, RX_ID=0xffff, SID=0xc1402)
fct_tpd_thread_4: scsitarget.hwpfct.dump.saved:notice]: A dump for adapter 3a was stored in /etc/log/fctsli_3a_1234/fct_fw_3a.dmp.gz.
- 交换机上的主机连接端口显示拥塞迹象
portstatsshow 47tim_txcrd_z 16483622 Time TX Credit Zero (2.5Us ticks)tim_txcrd_z_vc 0- 3: 0 0 0 0 tim_txcrd_z_vc 4- 7: 0 16483622 0 0 tim_txcrd_z_vc 8-11: 0 0 0 0 tim_txcrd_z_vc 12-15: 0 0 0 0 tim_txcrd_z_vc 16-19: 0 0 0 0 tim_txcrd_z_vc 20-23: 0 0 0 0 tim_txcrd_z_vc 24-27: 0 0 0 0 tim_txcrd_z_vc 28-31: 0 0 0 0 phy_stats_clear_ts 07-09-2026 GMT Thu 11:20:16 Timestamp of phy_port stats clearlgc_stats_clear_ts 07-09-2026 GMT Thu 11:20:16 Timestamp of lgc_port stats clearfec_corrected_rate 0 FEC Corrected blocks per second
- 在主机上可能会看到更高的 IO 等待时间、路径错误和超时
- 对于 VMWare:
NMP: nmp_ThrottleLogForDevice:2349: Cmd 0x1a(0x413680446300, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: H:0x0D:0x28 P:0x0 Valid sense data: 0x0 0x0 0x0. Act:NONE
NMP: nmp_ThrottleLogForDevice:2349: Cmd 0x88(0x412e80451080, 36817) to dev "naa.600a123" on path"vmhba1:C0:T1:L34" Failed: H:0x0 D:0x8 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL
- 对于 Windows:
454,Warning,WindowsHost1,ontapdsm,61141,"IO error: SCSI Queue FULL reported on LUN 0 on Path Id 03000002. The IO will be retried."
- 对于 RHEL:
Aug 21 10:55:31 hostname kernel: qla2xxx [0000:0e:00.0]-3820:1: QUEUE FULL detected