在进行 EC 重新平衡后,可能会报告 OLST ,而磁盘上存在对象块
适用场景
- StorageGRID 11.5
- ONTAP
- FabricPool
问题描述
以下文档介绍 了如何调查丢失的对象:
# grep OLST audit.log
2022-02-17T11:00:40.208953 [AUDT:[CBID(UI64):0x8504E653A4550568][UUID(CSTR):"F4A19BD8-8912-4544-B423-26A33E10F7C0"][PATH(CSTR):"ec-bucket/6310eec2-8483-4498-b07a-3bedfce2fff7/0808d4d1_00000000025cfef7_6310eec2-8483-4498-b07a-3bedfce2fff7"][RSLT(FC32):NONE][AVER(UI32):10][ATIM(UI64):1645095640208953][ATYP(FC32):OLST][ANID(UI32):12597448][AMID(FC32):ILMX][ATID(UI64):15044298786699829299]]
# telnet localhost 1402
ade 12448208: / > /proc/OBRP/ObjectByUUID F4A19BD8-8912-4544-B423-26A33E10F7C0
{
"TYPE": "CTNT",
"CHND": "1FEC9E09-23CC-4E54-B9E3-099482A8C62A",
"NAME": "6310eec2-8483-4498-b07a-3bedfce2fff7/0808d4d1_00000000025cfef7_6310eec2-8483-4498-b07a-3bedfce2fff7",
"CBID": "0x80D54C8D8B62B9D0",
"PHND": "3058F539-BCE5-11E9-9E53-650300B972EC",
"PPTH": "ec-bucket",
"META": {
"BASE": {
"ISIA": "10.64.203.173",
"PHTP": "1",
"PAWS": "2",
"ACCT": "12969787153409237262",
"BKAC": "12969787153409237262",
"*ctp": "application/x-binary",
},
"AWS3": {
"USDM": "{\"s3b-last-modified\":[\"20190808T092909Z\"],\"sha256\":[\"7f3ceaf54bcff0f63716900f6769b93597aae33e188e88792aa484a93d056dc9\"]}",
"LOCC": "us-east-1"
},
"SEGM": {
"SHSH": "MD5D 0x075B7CDE47E6140E4F4EC089FC2285A52D3400",
"CSIZ": "29012864"
},
"BYCB": {
"CTME": "1565603232952248",
"SHSH": "MD5D 0x1F6225B9C340CDE85ADDBFC5CC818590",
"CSIZ": "210",
"BSIZ": "1168",
"CVER": "196612",
"CFLG": "256",
"CTYP": "NONE",
"CHSH": "SHA1 0x537C146DE8137717EA037AD9DF43779E2E6B63D8",
"MTME": "1565603232944791",
"ITME": "1565603232952248"
},
"CMSM": {
"OWNR": "13461290",
"LATM": "1565603232952248"
}
}
}
请注意 CLCO:
,缺少 "" (位置)部分。
在 ONTAP 端,这可能表现为针对以下项触发了 ASUP :
HA Group Notification (WAFL INCONSISTENT USER DATA BLOCK) ALERT
或 ONTAP EMS 日志中出现类似以下内容的错误:
Thu Feb 17 12:00:39 +0100 [nodename: wafl_cbin_cloud_obj_create_pr: wafl.raid.incons.set.err.obj:alert]: Data block at VBN <VBN number> tiered to capacity tier (objid <object id>, slot <slot id>, btuuid <>, seqno <sequence number>) and marked with error (120). Block previously had error (120).
Thu Feb 17 12:08:48 +0100 [nodename: wafl_exempt08: wafl.raid.incons.userdata:error]: WAFL inconsistent: inconsistent user data block at VBN <VBN number> (vvbn:<vvbn number> fbn:<fbn> level:0) in public inode (fileid:<file id> snapid:0 file_type:1 disk_flags:0x8002 error:120 raid_set:1) in volume <volumename>@vserver:<vserver id>.
由于 ONTAP 发现对象存储中缺少块,因此 RAID 会将这些块标记为伪错误(错误: 120 )。