跳转到主内容

iSCSI 吞吐量低于预期

Views:
26
Visibility:
Public
Votes:
0
Category:
ontap-9
Specialty:
san
Last Updated:

适用场景

  • Cisco Unified Computing System ( UCS )
  • iSCSI
  • MPIO

问题描述

  • 在应用程序中观察 ~100 MB/ 秒( LVM )到 200 (单个 LUN ) MB/ 秒的吞吐量,而不是预期的 ~1GB/ 秒吞吐量,并在将 LUN 迁移到 LVM (条带化)配置中的新存储系统后使用性能测试工具 FiO 验证是否发生与应用程序无关的情况。
  • LVM
fio -thread -rw=readwrite -rwmixread=50 -bs=64k -size=20G -name=mytest --filename=/xxx/xxx/test1/file1
READ: bw=93.7MiB/s (98.3MB/s), 93.7MiB/s-93.7MiB/s (98.3MB/s-98.3MB/s), io=10.0GiB (10.7GB), run=109260-109260msec
WRITE: bw=93.7MiB/s (98.2MB/s), 93.7MiB/s-93.7MiB/s (98.2MB/s-98.2MB/s), io=9.00GiB (10.7GB), run=109260-109260msec
 
  • 单个 LUN
fio -thread-rw=readwrite -rwmixread=50 -bs=64k -size=20G -name=mytest --filename=/xxx/xxx/test2/file1
READ: bw=207MiB/s (217MB/s), 207MiB/s-207MiB/s (217MB/s-217MB/s), io=10.0GiB (10.7GB), run=49574-49574msec
WRITE: bw=207MiB/s (217MB/s), 207MiB/s-207MiB/s (217MB/s-217MB/s), io=9.00GiB (10.7GB), run=49574-49574msec

 
 
  • 受影响的配置(multipath -ll ):
xxxxxxxxxxxxx dm-8 NETAPP ,LUN C-Mode
size=400G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 4:0:0:5 sdab 65:176 active ready running
| |- 2:0:0:5 sdac 65:192 active ready running
| |- 7:0:0:5 sdad 65:208 active ready running
| `- 5:0:0:5 sdar 66:176 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
|- 0:0:0:5 sdf 8:80 active ready running
|- 1:0:0:5 sdl 8:176 active ready running
|- 3:0:0:5 sdam 66:96 active ready running
`- 6:0:0:5 sdav 66:240 active ready running
 
  • 跟踪时,以下 tcpdump 筛选器会捕获丢弃的数据:
    • tcpdump :

# tcpdump -i ibft0 -w [path]ibft0.pcap

  • 选择性确认:

tcp.options.sack.count > 0

  • 重复确认,(快速)重新传输,无序帧:
tcp.analysis.duplicate_ack || tcp.analysis.retransmission || tcp.analysis.fast_retransmission || tcp.analysis.out_of_order
  • 在 UCS 命令行界面( nxos )上,命令 show queuing interface ethernet x/xx (受影响的接口分别适用于相关 LUN )指示入站丢弃,这表示 NIC 缓冲区耗尽:
ucs-sas-A(nxos)# sh queuing interface ethernet 1/14

Ethernet1/14 queuing information:
[...]
RX Queuing
[...]

   qos-group 4
   q-size: 29760, HW MTU: 9216 (9216 configured)
   drop-type: drop, xon: 0, xoff: 29760
   Statistics:
     Pkts received over the port        : 16504320
     Ucast pkts sent to the cross-bar     : 16453589
     Mcast pkts sent to the cross-bar     : 0
     Ucast pkts received from the cross-bar  : 10259502
     Pkts sent to the port           : 10259502
     Pkts discarded on ingress         : 50731
     Per-priority-pause status         : Rx (Inactive), Tx (Inactive)
 
   qos-group 5
   q-size: 29760, HW MTU: 9216 (9216 configured)
   drop-type: drop, xon: 0, xoff: 29760
   Statistics:
     Pkts received over the port        : 50303345
     Ucast pkts sent to the cross-bar     : 49818733
     Mcast pkts sent to the cross-bar     : 0
     Ucast pkts received from the cross-bar  : 2162905
     Pkts sent to the port           : 2162905
    Pkts discarded on ingress         : 484613
     Per-priority-pause status         : Rx (Inactive), Tx (Inactive)

 

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.