CONAP-364039：意外检测信号丢失、导致FAS8300或AFF A400 MetroCluster IP系统发生接管事件
问题描述
- 报告了频繁的检测信号丢失、这可能导致节点接管。
- 正在进行接管的节点将在检测信号未命中之前无响应、在此期间不会记录任何节点日志。
- 发生接管的无响应节点在问题描述时间前后不会报告任何硬件错误。
- 接管的配对节点通常会记录SCSI IO传输错误、NVMM镜像同步失败、QP错误、HA错误等
- 配对节点日志示例：
Jan 01 01:01:01 [node_name: DR_heartbeat_thread: cf.ic.xferTimedOut:error]: HA interconnect: MCC_DRSOM transfer timed out.
Jan 01 01:01:01 [node_name: DR_heartbeat_thread: rdma.rlib.event.error:debug]: QP drsom_hb event error: client disconnect.
Jan 01 01:01:01 [node_name: iw_cxgbe: nvmm.mirror.aborting:debug]: mirror of sysid 1, partner_type HA Partner and mirror state NVMM_MIRROR_ONLINE is aborted because of reason NVMM_ERR_MIRROR_COMPLETION.
Jan 01 01:01:01 [node_name: rastrace_dump: rastrace.dump.saved:debug]: A RAS trace dump for module IC instance 0 was stored in /etc/log/rastrace/IC_0_20250101_01:01:01:123456.dmp.
Jan 01 01:01:01 [node_name: cf_firmware: cf.ic.xferTimedOut:error]: HA interconnect: OFW transfer timed out.
Jan 01 01:01:03 [node_name: cf_main: cf.fsm.takeoverByPartnerDisabled:error]: Failover monitor: takeover of node_name-01 by node_name-02 disabled (HA interconnect error. Verify that the partner node is running and that the HA interconnect cabling is correct, if applicable. For further assistance, contact technical support).
Jan 01 01:01:04 [node_name: cf_main: cf.fsm.partnerNotResponding:notice]: Failover monitor: partner not responding
Jan 01 01:02:01 [node_name: cf_main: cf.fsm.takeoverCountdown:info]: Failover monitor: takeover scheduled in 10 seconds
Jan 01 01:02:10 [node_name: cf_main: cf.fsm.takeover.noHeartbeat:alert]: Failover monitor: Takeover initiated after no heartbeat was detected from the partner node.