ONTAP集群节点意外重新启动:进程mgwd无响应11515秒(mgwd启动:"(2282)")、正在处理nodewdogg
适用场景
ONTAP 9
问题描述
- 节点意外重新启动:
Process mgwd unresponsive for 11515 seconds (mgwd startup: "(2282)") in process nodewatchdog on release 9.7P22
- 在节点重新启动之前、事件日志会显示SVM根卷已满:
Wed Aug 21 12:19:40 +0900 [Node-01: wafl_exempt04: wafl.vol.full:alert]: Insufficient space on volume svm1_root@vserver:vserver-uuid to perform operation. 116MB was requested but only 75.0MB was available.
Wed Aug 21 12:29:40 +0900 [Node-01: wafl_exempt00: wafl.vol.full:alert]: Insufficient space on volume svm1_root@vserver:vserver-uuid to perform operation. 4.00KB was requested but only 1.00KB was available.
Wed Aug 21 12:39:40 +0900 [Node-01: wafl_exempt05: wafl.vol.full:alert]: Insufficient space on volume svm1_root@vserver:vserver-uuid to perform operation. 4.00KB was requested but only 1.00KB was available.
Wed Aug 21 12:45:29 +0900 [Node-01: wafl_exempt03: wafl.memory.statusLowMemory:notice]: WAFL is running low on memory, with 938MB remaining.
Wed Aug 21 14:29:50 +0900 [Node-01: wafl_exempt06: wafl.vol.full:alert]: Insufficient space on volume svm1_root@vserver:vserver-uuid to perform operation. 4.00KB was requested but only 1.00KB was available.
Wed Aug 21 14:39:53 +0900 [Node-01: wafl_exempt06: wafl.vol.full:alert]: Insufficient space on volume svm1_root@vserver:vserver-uuid to perform operation. 4.00KB was requested but only 1.00KB was available.
Thu Aug 22 00:44:02 +0900 [Node-01: wafl_exempt02: wafl.memory.statusVeryLowMemory:alert]: WAFL is running very low on memory, with 634MB remaining.
Thu Aug 22 00:56:25 +0900 [Node-01: wafl_exempt05: wafl.memory.statusVeryLowMemory:alert]: WAFL is running very low on memory, with 636MB remaining.
DF显示SVM根卷中没有可用空间:
Filesystem kbytes used avail capacity Mounted on
/vol/svm1_root/ 72525891504 70495801660 2030089844 97% /vol/svm1_root/
/vol/svm1_root/.snapshot 3817152184 10975695064 0 288% /vol/svm1_root/.snapshot
cifs-share显示SVM根卷已共享给用户:
Vserver Share CIFS Server NetBIOS Name Path
svm1 01UserDataServer1 SH001084 /share/01UserDataServer1
svm1 02UserDataServer2 SH001084 /share/02UserDataServer2
...