MetroCluster IP远程站点多个磁盘出现故障

适用场景

问题描述

• 已在集群交换机的MetroCluster IP端口上禁用流量控制。
• 报告Multiple Disk Failure Event：HA Group Notification from Cluster1-1a (Files十二月：文件系统磁盘未响应)错误。
• 我们可以在集群中看到以下错误

NV镜像在集群网络碎片化警报发出前几秒钟即脱机

Mon Sep 11 15:03:37 +1000 [Cluster1-1a: nvmm_error: nvmm.mirror.offlined:debug]: params: {'mirror': 'HA_PARTNER'}
Mon Sep 11 15:03:37 +1000 [Cluster1-1a: nvmm_error: nvmm.mirror.offlined:debug]: params: {'mirror': 'DR_PARTNER'}

Mon Sep 11 15:03:45 +1000 [Cluster1-1a: vifmgr: vifmgr.port.monitor.failed:debug]: The "link_flapping" health check for port e0c (node Cluster1-1a) has failed. The port is operating in a degraded state.
Mon Sep 11 15:03:45 +1000 [Cluster1-1a: vifmgr: callhome.clus.net.degraded:debug]: Call home for CLUSTER NETWORK DEGRADED: Frequent Link Flapping - Cluster port e0c on node Cluster1-1a has experienced multiple link down notification

NV镜像状态将在一段时间后更改为联机

Mon Sep 11 15:15:44 +1000 [Cluster1-1a: nvmm_mirror_sync: nvmm.mirror.state.change:debug]: mirror of sysid 2, partner_type DR PARTNER, changed state from NVMM_MIRROR_SYNCING_OTHER to NVMM_MIRROR_ONLINE and took 1684 msecs.
Mon Sep 11 15:17:09 +1000 [Cluster1-1a: nvmm_mirror_sync: nvmm.mirror.state.change:debug]: mirror of sysid 2, partner_type DR PARTNER, changed state from NVMM_MIRROR_SYNCING_OTHER to NVMM_MIRROR_ONLINE and took 1605 msecs.

Mon Sep 11 15:12:53 +1000 [Cluster1-1b: nvmm_mirror_sync: nvmm.mirror.state.change:debug]: mirror of sysid 2, partner_type DR PARTNER, changed state from NVMM_MIRROR_SYNCING_OTHER to NVMM_MIRROR_ONLINE and took 1540 msecs.
Mon Sep 11 15:12:55 +1000 [Cluster1-1b: nvmm_mirror_sync: nvmm.mirror.state.change:debug]: mirror of sysid 1, partner_type HA Partner, changed state from NVMM_MIRROR_SYNCING_OTHER to NVMM_MIRROR_ONLINE and took 1545 msecs

• 部分或所有远程镜像丛已脱机、驱动器标记为出现故障。

Plex /Cluster1-1a_ssd_aggr1/plex1 (offline, failed, inactive, pool1)
RAID group /Cluster1-1a_ssd_aggr1/plex1/rg0 (partial)

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
dparity FAILED N/A 3630753/ -
parity FAILED N/A 3630753/ -
data FAILED N/A 3630753/ -
data FAILED N/A 3630753/ -
data FAILED N/A 3630753/ -
data FAILED N/A 3630753/ -
data FAILED N/A 3630753/ -
data FAILED N/A 3630753/ -
data FAILED N/A 3630753/ -
data FAILED N/A 3630753/ -
data FAILED N/A 3630753/ -
Raid group is missing 11 disks.

Plex /Cluster1-1a_root/plex12 (offline, failed, inactive, pool1)
RAID group /Cluster1-1a_root/plex12/rg0 (partial)

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
dparity FAILED N/A 63849/ -
parity FAILED N/A 63849/ -
data FAILED N/A 63849/ -
data FAILED N/A 63849/ -
data FAILED N/A 63849/ -
Raid group is missing 5 disks.

站点A：Cluster2

节点：
CLUSter2-1a—不存在问题
Cluster2-1b—不存在问题

站点B：Cluster1

节点：Cluster1-1a
-- ->使所有远程磁盘出现故障/缺少
Cluster1-1b -->没有问题

• 存储和交换机上没有底层硬件问题。