在具有动态阈值的 ONTAP 版本中报告可更正的内存错误

适用于

ONTAP 版本：

• 9.5 及更高主要版本
• 9.4P6 及更高版本 P 发布
• 9.3P11 及更高版本 P 发布
• 9.1P18 及更高版本 P 发布

平台：

• AFF 系统
• ASA 系统
• FAS 系统

• 关键CECC计数内存错误警报

注意：对于所有其他 ONTAP 版本，请参阅如何排除 FAS 和 AFF 系统上的可更正内存错误

问题描述

• 运行状况警报 AutoSupport 消息：

HA Group Notification (Health Monitor process nphm: CriticalCECCCountMemErrAlert[DIMM-16]) ALERT

• EMS 报告：

[mgwd: callhome.hm.alert.critical:alert]: Call home for Health Monitor process nphm: CriticalCECCCountMemErrAlert[DIMM-16].

• 管理日志

[nphmd: hm.alert.raised:alert]: Alert Id = CriticalCECCCountMemErrAlert , Alerting Resource = DIMM-16 raised by monitor controller

• 运行状况监控报告

::> system health alert show
        Node: Node1
      Alert ID: CriticalCECCCountMemErrAlert
      Resource: DIMM-1
      Severity: Critical
   Indication Time: Wed Jan 10 16:45:18 2024
      Suppress: false
     Acknowledge: false
   Probable Cause: The DIMM has degraded, leading to memory errors.
  Possible Effect: Memory issues can lead to a catastrophic system panic,
           which can lead to data downtime on the node.
 Corrective Actions:
 1. Contact technical support to obtain a new DIMM of the same specification.
 2. If possible, perform a takeover of this node and bring the node down for maintenance.
 3. Refer to the DIMM replacement guide for your given hardware platform to replace the DIMM.
 4. Bring the storage system online.

Alertdditional Information  
 Slot Name: DIMM-16
 CPU Socket: 0
 Channel: 0
 Slot Number on a Channel: 0
 Correctable ECC Error Count: 329793
 Uncorrectable ECC Error Count: 0
 Correctable ECC Error Limit: 0
 CECC Alert Method: bucket
 Correctable ECC Replace DIMM: true
 Failure Reason: none
 Correctable ECC Error Minimum Count: 325104
 Correctable ECC Error Bucket Count: 177243
 Correctable ECC Replace DIMM Count: 3
 DIMM Serial Number: *****
 Node Serial Number: *****
 Node Model: FAS****
 Alerting Resource Name   DIMM-16

• 内存 DIMM 显示将 DIMM 状态报告为已降级

Cluster::>memory dimm show
              DIMM     UECC  CECC  Alert    CPU           Slot          Failure
Node          Name    Count Count Method Socket Channel Number Status    Reason
------------- ------- ----- ----- ------ ------ ------- ------ ------- --------
Cluster-01
              DIMM-1      0     0 bucket      1       0      0 ok          none
              DIMM-2      0     0 bucket      1       0      1 ok          none
              DIMM-3      0     0 bucket      1       1      0 ok          none
              DIMM-4      0     0 bucket      1       1      1 ok          none
              DIMM-5      0     0 bucket      1       3      1 ok          none
              DIMM-6      0     0 bucket      1       3      0 ok          none
              DIMM-7      0     0 bucket      1       2      1 ok          none
              DIMM-8      0     0 bucket      1       2      0 ok          none
              DIMM-9      0     0 bucket      0       2      0 ok          none
              DIMM-10     0     0 bucket      0       2      1 ok          none
              DIMM-11     0     0 bucket      0       3      0 ok          none
              DIMM-12     0     0 bucket      0       3      1 ok          none
              DIMM-13     0     0 bucket      0       1      1 ok          none
              DIMM-14     0     0 bucket      0       1      0 ok          none
              DIMM-15     0     0 bucket      0       0      1 ok          none
              DIMM-16     0     0 bucket      0       0      0 degraded   none
 

• 当在 AutoSupport 日志中的 sktrace 中看到错误时，可能会出现性能问题

CECC_LOG_ENTRY: Number of correctable ECC since boot 355314194: Information about Correctable ECC: ECC error at DIMM 8: CE-12-1234-12345678,ADDR 0xaada0eec0, (Node(0), Memory controller(1), CH(2), DIMM(0), Rank(0), Bank Group(3), Bank(0x2), Row(0x2174), Col(0x1d8))