节点关闭,多个风扇出现故障
适用场景
- AFF A300 , AFF A200 , FAS8200 , FAS2650 , FAS2620 ( SP )
- AFF A220 , AFF C190 , FAS2750 , FAS2720 ( BMC )
问题描述
- 控制器因多个风扇故障而关闭。
env_mgr: monitor.chassisFan.stop:error]: Chassis fan contains at least one stopped fan: SysFan3 F1 (1260 RPM)
env_mgr: monitor.chassisFan.stop:error]: Chassis fan contains at least one stopped fan: SysFan3 F2 (1260 RPM)
env_mgr: monitor.chassisFanFail.xMinShutdown:EMERGENCY]: Multiple Chassis Fan failure: System will shut down in 2 minutes.
monitor: monitor.globalStatus.critical:EMERGENCY]: Multiple fans has failed: Sysfan3 F1, Sysfan3 F2.
statd: monitor.fan.failed:alert]: Multiple fans has failed: Sysfan3 F1, Sysfan3 F2.
statd: monitor.shutdown.emergency:EMERGENCY]: Emergency shutdown: Environmental Reason Shutdown (Multiple fans failed)
sysconfig -M
该风扇的输出:
Fri Jun 18 2021, 23:48:28 MSK !FAN3!021819020448!441-00025!A1!
Sat Jun 19 2021, 01:04:18 MSK !FAN3: Error reading FRU EEPROM
Sat Jun 19 2021, 07:05:27 MSK !FAN3!021819020448!441-00025!A1!
- 更换风扇后 EMS- 问题描述仍保持不变:
env_mgr: monitor.chassisFan.removed:alert]: Chassis fan SysFan3 is removed
env_mgr: monitor.chassisFan.ok:notice]: Chassis fan Sysfan3 Present is ok.
env_mgr: monitor.chassisFan.ok:notice]: Chassis fan Sysfan3 F1 is ok.
env_mgr: monitor.chassisFan.ok:notice]: Chassis fan Sysfan3 F2 is ok.
env_mgr: monitor.chassisFan.stop:error]: Chassis fan contains at least one stopped fan: SysFan3 F1 (failed)
env_mgr: monitor.chassisFan.stop:error]: Chassis fan contains at least one stopped fan: SysFan3 F2 (failed)
env_mgr: monitor.chassisFanFail.xMinShutdown:EMERGENCY]: Multiple Chassis Fan failure: System will shut down in 2 minutes.
monitor: monitor.globalStatus.critical:EMERGENCY]: Multiple fans has failed: Sysfan3 F1, Sysfan3 F2.
statd: monitor.shutdown.emergency:EMERGENCY]: Emergency shutdown: Environmental Reason Shutdown (Multiple fans failed)
statd: monitor.shutdown.emergency:EMERGENCY]: Emergency shutdown: Environmental Reason Shutdown (Multiple fans failed)
shutdown_thread0: ha.localNodeShutDown:notice]: Shutdown of the local node has been initiated with inhibit_takeover set to FALSE.
cf_main: cf.fsm.takeoverOfPartnerDisabled:error]: Failover monitor: takeover of cluster2-01 disabled (local halt in progress).
shutdown_thread0: kern.shutdown:notice]: System shut down because : "Environmental Shutdown".
Wed Jun 23 20:46:54 MSK [toaster1: mgwd: callhome.hm.alert.critical:alert]: Call home for Health Monitor process cphm: CriticalFanFruFaultAlert[021604000131].
Thu Jun 24 11:00:00 MSK [toaster1: statd: monitor.fan.failed:alert]: Multiple fans has failed: Sysfan3 F1, Sysfan3 F2.
Thu Jun 24 11:18:36 MSK [toaster1: env_mgr: monitor.chassisFan.stop:error]: Chassis fan contains at least one stopped fan: SysFan3 F1 (failed)
Thu Jun 24 11:18:37 MSK [toaster1: env_mgr: monitor.chassisFan.stop:error]: Chassis fan contains at least one stopped fan: SysFan3 F2 (failed)
Thu Jun 24 11:19:06 MSK [toaster1: env_mgr: callhome.c.fan.fru.fault:error]: Call home for CHASSIS FAN FRU FAILED: SysFan3 F1
Thu Jun 24 11:19:06 MSK [toaster1: env_mgr: callhome.c.fan.fru.fault:error]: Call home for CHASSIS FAN FRU FAILED: SysFan3 F2
Thu Jun 24 11:00:00 MSK [toaster2: statd: monitor.fan.failed:alert]: Multiple fans has failed: Sysfan3 F1, Sysfan3 F2.
Thu Jun 24 11:53:52 MSK [toaster2: env_mgr: monitor.chassisFan.stop:error]: Chassis fan contains at least one stopped fan: SysFan3 F1 (failed)
Thu Jun 24 11:53:52 MSK [toaster2: env_mgr: monitor.chassisFan.stop:error]: Chassis fan contains at least one stopped fan: SysFan3 F2 (failed)
Oct 25 09:15:45 [toaster1:monitor.chassisFanFail.xMinShutdown:EMERGENCY]: Multiple Chassis Fan failure: System will shut down in 2 minutes.
Oct 25 09:16:00 [toaster1:monitor.globalStatus.critical:EMERGENCY]: Multiple fans has failed: Sysfan3 F2, Sysfan3 F1.
login: Oct 25 09:16:56 [toaster1:object.store.unavailable:EMERGENCY]: Unable to connect to the object store "ndas_rashidn12" from node d654735a-7cc4-11e9-9553-00a0985a07aa. Reason: Access denied.
Oct 25 09:18:16 [toaster:monitor.shutdown.emergency:EMERGENCY]: Emergency shutdown: Environmental Reason Shutdown (Multiple fans failed)