Hadoop应用程序查询无法执行并写入StorageGRID存储分段、并发生超时
适用场景
- StorageGRID 11.6.0.13
- Hadoop
问题描述
- 由于Hadoop应用程序查询无法写入存储分段、因此无法运行并超时。
- 源应用程序错误：
botocore.exceptions.Client
2024-01-24 17:11:54.335 DEBUG com.amazonaws.thirdparty.apache. http.wire: 'main': http-outgoing-1 <<"<Error><Code>ServiceUnavailable</Code><Message>Please reduce your request rate.</Message><Resource>/<BUCKET>/?uploads&prefix=daXXXrmediate%2Fquery_test2%2F</Resource><RequestId></RequestId></Error>"
2024-01-24 17:11:54.335 DEBUG com.amazonaws.thirdparty.apache. http.headers: 'main': http-outgoing-1 << HTTP/1.1 503 Service Unavailable- 从StorageGRID 日志中：
Jan 25 08:46:07 XXXNODE04 ADE: |12330832 2577485089 S3RQ %CEA 2024-01-25T08:46:07.470023| NOTICE 1448 3018ddf97d08def3 S3RQ: EVENT_PROCESS_CREATE - connection=17061XXX127 method=GET name=</<BUCKET>/?uploads&prefix=datalake_intermediate%2Fquery_test%2F> auth=<V4> clientIP=<>
Jan 25 08:47:01 XXXNODE04 ADE: |12330832 2577485089 S3RQ CSEN 2024-01-25T08:47:01.869914| WARNING 0236 3018ddf97d08def3 S3RQ: S3Request_Module::processCreate Cassandra TimeoutException (posSuc: Failed to execute statement while paging. - Cassandra Driver Error(Request timed out):'Request timed out' Detailed Info:[N/A])
Jan 25 08:47:01 XXXNODE04 ADE: |12330832 2577485089 S3RQ CSEN 2024-01-25T08:47:01.869996| NOTICE 0089 3018ddf97d08def3 S3RQ: S3 error response: RequestId=1706XXX65127, TraceId=3018ddf97d08def3, Resource=/<BUCKET>/?uploads&prefix=daXXXmediate%2Fquery_test%2F, HTTP Method=GET, HTTP Status Code=503, X-Forwarded-For: 'XX.XX.XXX.91', ErrorMsg=ServiceUnavailable, ErrorType=Internal, CustomErrorMessage={<none>}, Details={posSuc: Failed to execute statement while paging. - Cassandra Driver Error(Request timed out):'Request timed out' Detailed Info:[N/A]}