From noreply@localhost.localdomain Thu Jul 18 16:50:23 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id CEBC042A143C for ; Thu, 18 Jul 2024 16:50:23 +0800 (CST) Date: Thu, 18 Jul 2024 16:50:23 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <2012132362.0.1721292623816@localhost> Subject: [Cloudera Alert] 3 Alerts since 4:49:49 PM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-602 [Cloudera Alert] 3 Alerts since 4:49:49 PM
Health test changes: 1 Became Bad, 1 Became Concerning, 5 Became Unknown
Time: Jul 18, 2024 4:49:49 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_DATA_NODES_HEALTHY Service health test bad Critical The health test result for HDFS_DATA_NODES_HEALTHY has become bad: Healthy DataNode: 0. Concerning DataNode: 0. Total DataNode: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
Health test changes: 2 Became Bad
Time: Jul 18, 2024 4:49:49 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_CANARY_HEALTH Service health test bad Critical The health test result for ZOOKEEPER_CANARY_HEALTH has become bad: The ZooKeeper service canary failed for an unknown reason.
ZOOKEEPER_SERVERS_HEALTHY Service health test bad Critical The health test result for ZOOKEEPER_SERVERS_HEALTHY has become bad: Healthy Server: 0. Concerning Server: 0. Total Server: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
The health test result for HDFS_CANARY_HEALTH has become bad: Canary test failed to create parent directory for /tmp/.cloudera_health_monitoring_canary_files.
Time: Jul 18, 2024 4:50:04 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_CANARY_HEALTH Service health test bad Critical The health test result for HDFS_CANARY_HEALTH has become bad: Canary test failed to create parent directory for /tmp/.cloudera_health_monitoring_canary_files.
From noreply@localhost.localdomain Thu Jul 18 16:55:23 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 9E6E542A17EB for ; Thu, 18 Jul 2024 16:55:23 +0800 (CST) Date: Thu, 18 Jul 2024 16:55:23 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <292444334.1.1721292923638@localhost> Subject: [Cloudera Alert] 7 Alerts since 4:54:17 PM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-1465 [Cloudera Alert] 7 Alerts since 4:54:17 PM
The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 4:54:17 PM
Monitor Startup: false
Role: Server (hadoop102)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for YARN_RESOURCEMANAGERS_HEALTH has become bad: ResourceManager summary: Hadoop102 (Availability: Active, Health: Bad). This health test reflects the health of the active ResourceManager.
Time: Jul 18, 2024 4:54:17 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_RESOURCEMANAGERS_HEALTH Service health test bad Critical The health test result for YARN_RESOURCEMANAGERS_HEALTH has become bad: ResourceManager summary: Hadoop102 (Availability: Active, Health: Bad). This health test reflects the health of the active ResourceManager.
The health test result for SECONDARY_NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 4:54:17 PM
Monitor Startup: false
Role: SecondaryNameNode (hadoop102)
Role Type: SecondaryNameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
SECONDARY_NAME_NODE_HOST_HEALTH Role health test bad Critical The health test result for SECONDARY_NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 4:54:17 PM
Monitor Startup: false
Role: NodeManager (hadoop102)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 4:54:17 PM
Monitor Startup: false
Role: ResourceManager (hadoop102)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for YARN_NODE_MANAGERS_HEALTHY has become bad: Healthy NodeManager: 2. Concerning NodeManager: 0. Total NodeManager: 3. Percent healthy: 66.67%. Percent healthy or concerning: 66.67%. Critical threshold: 90.00%.
Time: Jul 18, 2024 4:54:22 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_NODE_MANAGERS_HEALTHY Service health test bad Critical The health test result for YARN_NODE_MANAGERS_HEALTHY has become bad: Healthy NodeManager: 2. Concerning NodeManager: 0. Total NodeManager: 3. Percent healthy: 66.67%. Percent healthy or concerning: 66.67%. Critical threshold: 90.00%.
The health test result for HDFS_DATA_NODES_HEALTHY has become bad: Healthy DataNode: 2. Concerning DataNode: 0. Total DataNode: 3. Percent healthy: 66.67%. Percent healthy or concerning: 66.67%. Critical threshold: 90.00%.
Time: Jul 18, 2024 4:54:22 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_DATA_NODES_HEALTHY Service health test bad Critical The health test result for HDFS_DATA_NODES_HEALTHY has become bad: Healthy DataNode: 2. Concerning DataNode: 0. Total DataNode: 3. Percent healthy: 66.67%. Percent healthy or concerning: 66.67%. Critical threshold: 90.00%.
From noreply@localhost.localdomain Thu Jul 18 16:59:23 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id A0C7742A17E5 for ; Thu, 18 Jul 2024 16:59:23 +0800 (CST) Date: Thu, 18 Jul 2024 16:59:23 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <816868183.2.1721293163643@localhost> Subject: [Cloudera Alert] The health of service yarn has become bad. MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-1806 [Cloudera Alert] The health of service yarn has become bad.
The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
Time: Jul 18, 2024 4:59:08 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_JOBHISTORY_HEALTH Service health test bad Critical The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
From noreply@localhost.localdomain Thu Jul 18 17:00:23 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 9A3BF42A17E5 for ; Thu, 18 Jul 2024 17:00:23 +0800 (CST) Date: Thu, 18 Jul 2024 17:00:23 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1782353558.3.1721293223630@localhost> Subject: [Cloudera Alert] The health of service hdfs has become bad. MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-2171 [Cloudera Alert] The health of service hdfs has become bad.
Health test changes: 1 Became Bad, 2 Became Concerning, 1 Became Good, 5 Became Unknown
Time: Jul 18, 2024 5:00:03 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVER_CONTROLLERS_HEALTHY Service health test bad Critical The health test result for HDFS_FAILOVER_CONTROLLERS_HEALTHY has become bad: Failover Controllers that are not running: Hadoop102, Hadoop101.
From noreply@localhost.localdomain Thu Jul 18 17:01:23 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 9739F42A17EF for ; Thu, 18 Jul 2024 17:01:23 +0800 (CST) Date: Thu, 18 Jul 2024 17:01:23 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1848965068.4.1721293283617@localhost> Subject: [Cloudera Alert] Health test changes: 5 Became Bad MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-2308 [Cloudera Alert] Health test changes: 5 Became Bad
Health test changes: 5 Became Bad
Time: Jul 18, 2024 5:00:48 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FREE_SPACE_REMAINING Service health test bad Critical The health test result for HDFS_FREE_SPACE_REMAINING has become bad: Test failed because there is no running NameNode: Test of whether HDFS has enough free space.
HDFS_BLOCKS_WITH_CORRUPT_REPLICAS Service health test bad Critical The health test result for HDFS_BLOCKS_WITH_CORRUPT_REPLICAS has become bad: Test failed because there is no running NameNode: Test of whether HDFS has too many blocks with at least one corrupt replica.
HDFS_MISSING_BLOCKS Service health test bad Critical The health test result for HDFS_MISSING_BLOCKS has become bad: Test failed because there is no running NameNode: Test of whether HDFS has too many missing blocks.
HDFS_UNDER_REPLICATED_BLOCKS Service health test bad Critical The health test result for HDFS_UNDER_REPLICATED_BLOCKS has become bad: Test failed because there is no running NameNode: Test of whether HDFS has too many under-replicated blocks.
HDFS_CANARY_HEALTH Service health test bad Critical The health test result for HDFS_CANARY_HEALTH has become bad: Test failed because there is no running NameNode: Test of whether basic HDFS operations work.
From noreply@localhost.localdomain Thu Jul 18 17:07:23 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 9E0CA42A17F2 for ; Thu, 18 Jul 2024 17:07:23 +0800 (CST) Date: Thu, 18 Jul 2024 17:07:23 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1391208747.5.1721293643633@localhost> Subject: [Cloudera Alert] The health of service yarn has become bad. MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-2973 [Cloudera Alert] The health of service yarn has become bad.
The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
Time: Jul 18, 2024 5:06:19 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_JOBHISTORY_HEALTH Service health test bad Critical The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
From noreply@localhost.localdomain Thu Jul 18 17:21:23 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id A6DB442A2B02 for ; Thu, 18 Jul 2024 17:21:23 +0800 (CST) Date: Thu, 18 Jul 2024 17:21:23 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <553141797.6.1721294483661@localhost> Subject: [Cloudera Alert] The health of service yarn has become bad. MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-4064 [Cloudera Alert] The health of service yarn has become bad.
The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
Time: Jul 18, 2024 5:21:07 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_JOBHISTORY_HEALTH Service health test bad Critical The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
From noreply@localhost.localdomain Thu Jul 18 17:33:23 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 9FF0842A0E42 for ; Thu, 18 Jul 2024 17:33:23 +0800 (CST) Date: Thu, 18 Jul 2024 17:33:23 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <18003563.7.1721295203641@localhost> Subject: [Cloudera Alert] 3 Alerts since 5:32:59 PM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-5143 [Cloudera Alert] 3 Alerts since 5:32:59 PM
The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The SPARK_YARN_HISTORY_SERVER is not running.
Time: Jul 18, 2024 5:32:59 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH Service health test bad Critical The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The SPARK_YARN_HISTORY_SERVER is not running.
The health test result for HDFS_FAILOVER_CONTROLLERS_HEALTHY has become bad: Failover Controllers that are not running: Hadoop102, Hadoop101.
Time: Jul 18, 2024 5:33:04 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVER_CONTROLLERS_HEALTHY Service health test bad Critical The health test result for HDFS_FAILOVER_CONTROLLERS_HEALTHY has become bad: Failover Controllers that are not running: Hadoop102, Hadoop101.
The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
Time: Jul 18, 2024 5:33:04 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_JOBHISTORY_HEALTH Service health test bad Critical The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
From noreply@localhost.localdomain Thu Jul 18 17:40:23 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id A2A4342A0E42 for ; Thu, 18 Jul 2024 17:40:23 +0800 (CST) Date: Thu, 18 Jul 2024 17:40:23 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <983703990.8.1721295623642@localhost> Subject: [Cloudera Alert] 3 Alerts since 5:39:36 PM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-5982 [Cloudera Alert] 3 Alerts since 5:39:36 PM
The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The SPARK_YARN_HISTORY_SERVER is not running.
Time: Jul 18, 2024 5:39:36 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH Service health test bad Critical The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The SPARK_YARN_HISTORY_SERVER is not running.
The health test result for HDFS_FAILOVER_CONTROLLERS_HEALTHY has become bad: Failover Controllers that are not running: Hadoop102, Hadoop101.
Time: Jul 18, 2024 5:39:41 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVER_CONTROLLERS_HEALTHY Service health test bad Critical The health test result for HDFS_FAILOVER_CONTROLLERS_HEALTHY has become bad: Failover Controllers that are not running: Hadoop102, Hadoop101.
The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
Time: Jul 18, 2024 5:39:41 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_JOBHISTORY_HEALTH Service health test bad Critical The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
From noreply@localhost.localdomain Thu Jul 18 17:41:23 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 99B6542A0EA8 for ; Thu, 18 Jul 2024 17:41:23 +0800 (CST) Date: Thu, 18 Jul 2024 17:41:23 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1257742468.9.1721295683627@localhost> Subject: [Cloudera Alert] 3 Alerts since 5:40:16 PM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-6125 [Cloudera Alert] 3 Alerts since 5:40:16 PM
Health test changes: 1 Became Bad, 1 Became Concerning, 7 Became Good
Time: Jul 18, 2024 5:40:16 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_HA_NAMENODE_HEALTH Service health test bad Critical The health test result for HDFS_HA_NAMENODE_HEALTH has become bad: NameNode summary: Hadoop101 (Availability: Active, Health: Bad), Hadoop102 (Availability: Standby, Health: Bad). This health test reflects the health of the active NameNode.
The health test result for NAME_NODE_SAFE_MODE has become bad: This NameNode is in safe mode.
Time: Jul 18, 2024 5:40:16 PM
Monitor Startup: false
Role: NameNode (hadoop101)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_SAFE_MODE Role health test bad Critical The health test result for NAME_NODE_SAFE_MODE has become bad: This NameNode is in safe mode.
The health test result for NAME_NODE_SAFE_MODE has become bad: This NameNode is in safe mode.
Time: Jul 18, 2024 5:40:16 PM
Monitor Startup: false
Role: NameNode (hadoop102)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_SAFE_MODE Role health test bad Critical The health test result for NAME_NODE_SAFE_MODE has become bad: This NameNode is in safe mode.
From noreply@localhost.localdomain Thu Jul 18 18:03:23 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 9E0C442A0EA2 for ; Thu, 18 Jul 2024 18:03:23 +0800 (CST) Date: Thu, 18 Jul 2024 18:03:23 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1361997605.10.1721297003632@localhost> Subject: [Cloudera Alert] 3 Alerts since 6:02:21 PM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-8068 [Cloudera Alert] 3 Alerts since 6:02:21 PM
The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
Time: Jul 18, 2024 6:02:21 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_JOBHISTORY_HEALTH Service health test bad Critical The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The SPARK_YARN_HISTORY_SERVER is not running.
Time: Jul 18, 2024 6:02:21 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH Service health test bad Critical The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The SPARK_YARN_HISTORY_SERVER is not running.
The health test result for HDFS_FAILOVER_CONTROLLERS_HEALTHY has become bad: Failover Controllers that are not running: Hadoop102, Hadoop101.
Time: Jul 18, 2024 6:02:26 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVER_CONTROLLERS_HEALTHY Service health test bad Critical The health test result for HDFS_FAILOVER_CONTROLLERS_HEALTHY has become bad: Failover Controllers that are not running: Hadoop102, Hadoop101.
From noreply@localhost.localdomain Thu Jul 18 18:12:23 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id ABED44085FA5 for ; Thu, 18 Jul 2024 18:12:23 +0800 (CST) Date: Thu, 18 Jul 2024 18:12:23 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1603005520.11.1721297543691@localhost> Subject: [Cloudera Alert] The health of service spark_on_yarn has become bad. MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-9135 [Cloudera Alert] The health of service spark_on_yarn has become bad.
The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The SPARK_YARN_HISTORY_SERVER is not running.
Time: Jul 18, 2024 6:12:05 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH Service health test bad Critical The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The SPARK_YARN_HISTORY_SERVER is not running.
From noreply@localhost.localdomain Thu Jul 18 18:15:41 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id D834C42A4E83 for ; Thu, 18 Jul 2024 18:15:41 +0800 (CST) Date: Thu, 18 Jul 2024 18:15:41 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1704627313.12.1721297741871@localhost> Subject: [Cloudera Alert] 32 Alerts since 6:15:24 PM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-9635 [Cloudera Alert] 32 Alerts since 6:15:24 PM
The health test result for HUE_SERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: Hue Server (hadoop102)
Role Type: Hue Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
HUE_SERVER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for HUE_SERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for NODE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: NodeManager (hadoop101)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NODE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for HDFS_HA_NAMENODE_HEALTH has become bad: NameNode summary: Hadoop101 (Availability: Active, Health: Bad), Hadoop102 (Availability: Standby, Health: Bad). This health test reflects the health of the active NameNode.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_HA_NAMENODE_HEALTH Service health test bad Critical The health test result for HDFS_HA_NAMENODE_HEALTH has become bad: NameNode summary: Hadoop101 (Availability: Active, Health: Bad), Hadoop102 (Availability: Standby, Health: Bad). This health test reflects the health of the active NameNode.
The health test result for JOURNAL_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: JournalNode (hadoop102)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for JOURNAL_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for NODE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: NodeManager (hadoop103)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NODE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for RESOURCE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: ResourceManager (hadoop103)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for RESOURCE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for NAME_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: NameNode (hadoop101)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NAME_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 1 Became Concerning, 1 Became Unknown
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: Impala Daemon (hadoop102)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_QUERY_MONITORING_STATUS Role health test bad Critical The health test result for IMPALAD_QUERY_MONITORING_STATUS has become bad: There are 2 error(s) seen monitoring executing queries, and 0 errors(s) seen monitoring completed queries for this role in the previous 5 minute(s). Critical threshold: any.
IMPALAD_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for IMPALAD_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_RESOURCEMANAGERS_HEALTH Service health test bad Critical The health test result for YARN_RESOURCEMANAGERS_HEALTH has become bad: ResourceManager summary: Hadoop102 (Availability: Active, Health: Bad), Hadoop103 (Availability: Unknown, Health: Bad). This health test reflects the health of the active ResourceManager.
YARN_JOBHISTORY_HEALTH Service health test bad Critical The health test result for YARN_JOBHISTORY_HEALTH has become bad: The health of the JobHistory Server is bad. The following health tests are bad: web server status.
The health test result for JOBHISTORY_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: JobHistory Server (hadoop103)
Role Type: JobHistory Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
JOBHISTORY_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for JOBHISTORY_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for HOST_MONITOR_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: Host Monitor (hadoop101)
Role Type: Host Monitor
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HOST_MONITOR_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for HOST_MONITOR_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Health Test Results:
Health Test Name Event Code Severity Content
IMPALA_STATESTORE_HEALTH Service health test bad Critical The health test result for IMPALA_STATESTORE_HEALTH has become bad: The health of the Impala StateStore is bad. The following health tests are bad: web server status.
IMPALA_CATALOGSERVER_HEALTH Service health test bad Critical The health test result for IMPALA_CATALOGSERVER_HEALTH has become bad: The health of the Impala Catalog Server is bad. The following health tests are bad: web server status.
The health test result for STATESTORE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: Impala StateStore (hadoop103)
Role Type: Impala StateStore
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
STATESTORE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for STATESTORE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for JOURNAL_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: JournalNode (hadoop103)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for JOURNAL_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for NODE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: NodeManager (hadoop102)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NODE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for JOURNAL_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: JournalNode (hadoop101)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for JOURNAL_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for SERVICE_MONITOR_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: Service Monitor (hadoop101)
Role Type: Service Monitor
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
SERVICE_MONITOR_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for SERVICE_MONITOR_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for RESOURCE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: ResourceManager (hadoop102)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for RESOURCE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 1 Became Concerning, 1 Became Unknown
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: Impala Daemon (hadoop101)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_QUERY_MONITORING_STATUS Role health test bad Critical The health test result for IMPALAD_QUERY_MONITORING_STATUS has become bad: There are 2 error(s) seen monitoring executing queries, and 0 errors(s) seen monitoring completed queries for this role in the previous 5 minute(s). Critical threshold: any.
IMPALAD_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for IMPALAD_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 1 Became Concerning, 1 Became Unknown
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: Impala Daemon (hadoop103)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_QUERY_MONITORING_STATUS Role health test bad Critical The health test result for IMPALAD_QUERY_MONITORING_STATUS has become bad: There are 1 error(s) seen monitoring executing queries, and 0 errors(s) seen monitoring completed queries for this role in the previous 5 minute(s). Critical threshold: any.
IMPALAD_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for IMPALAD_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for NAME_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: NameNode (hadoop102)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NAME_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 1 Became Unknown
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: Impala Catalog Server (hadoop102)
Role Type: Impala Catalog Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
CATALOGSERVER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for CATALOGSERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for EVENT_SERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Time: Jul 18, 2024 6:15:24 PM
Monitor Startup: false
Role: Event Server (hadoop101)
Role Type: Event Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
EVENT_SERVER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for EVENT_SERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Server (hadoop102)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HIVEMETASTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Hive Metastore Server (hadoop101)
Role Type: Hive Metastore Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HIVEMETASTORE_HOST_HEALTH Role health test bad Critical The health test result for HIVEMETASTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
The health test result for HUE_HUE_SERVERS_HEALTHY has become bad: Healthy Hue Server: 0. Concerning Hue Server: 0. Total Hue Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Health Test Results:
Health Test Name Event Code Severity Content
HUE_HUE_SERVERS_HEALTHY Service health test bad Critical The health test result for HUE_HUE_SERVERS_HEALTHY has become bad: Healthy Hue Server: 0. Concerning Hue Server: 0. Total Hue Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
The health test result for HUE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Hue Server (hadoop102)
Role Type: Hue Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
HUE_SERVER_HOST_HEALTH Role health test bad Critical The health test result for HUE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: NodeManager (hadoop101)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: kafka_broker (hadoop102)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HOST_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Health test changes: 2 Became Bad
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_DATA_NODES_HEALTHY Service health test bad Critical The health test result for HDFS_DATA_NODES_HEALTHY has become bad: Healthy DataNode: 0. Concerning DataNode: 0. Total DataNode: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
HDFS_FAILOVER_CONTROLLERS_HEALTHY Service health test bad Critical The health test result for HDFS_FAILOVER_CONTROLLERS_HEALTHY has become bad: Failover Controllers with bad health: Hadoop102, Hadoop101. The following health tests are bad: host health. The following health tests are bad: host health.
The health test result for OOZIE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Oozie Server (hadoop102)
Role Type: Oozie Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: oozie
Service Display Name: Oozie
Service Type: Oozie
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
OOZIE_SERVER_HOST_HEALTH Role health test bad Critical The health test result for OOZIE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Failover Controller (hadoop101)
Role Type: Failover Controller
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVERCONTROLLER_HOST_HEALTH Role health test bad Critical The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
From noreply@localhost.localdomain Thu Jul 18 18:15:41 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id DCE2142A4E84 for ; Thu, 18 Jul 2024 18:15:41 +0800 (CST) Date: Thu, 18 Jul 2024 18:15:41 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <603834013.13.1721297741903@localhost> Subject: [Cloudera Alert] 32 Alerts since 6:15:29 PM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-9835 [Cloudera Alert] 32 Alerts since 6:15:29 PM
The health test result for SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: spark_yarn_history_server (hadoop103)
Role Type: SPARK_YARN_HISTORY_SERVER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HOST_HEALTH Role health test bad Critical The health test result for SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: JournalNode (hadoop102)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_HOST_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HIVESERVER2_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: HiveServer2 (hadoop101)
Role Type: HiveServer2
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HIVESERVER2_HOST_HEALTH Role health test bad Critical The health test result for HIVESERVER2_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: NodeManager (hadoop103)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: ResourceManager (hadoop103)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: NameNode (hadoop101)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_HOST_HEALTH Role health test bad Critical The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Impala Daemon (hadoop102)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_HOST_HEALTH Role health test bad Critical The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for YARN_NODE_MANAGERS_HEALTHY has become bad: Healthy NodeManager: 0. Concerning NodeManager: 0. Total NodeManager: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_NODE_MANAGERS_HEALTHY Service health test bad Critical The health test result for YARN_NODE_MANAGERS_HEALTHY has become bad: Healthy NodeManager: 0. Concerning NodeManager: 0. Total NodeManager: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
The health test result for JOBHISTORY_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: JobHistory Server (hadoop103)
Role Type: JobHistory Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
JOBHISTORY_HOST_HEALTH Role health test bad Critical The health test result for JOBHISTORY_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Server (hadoop101)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
The health test result for HOST_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Host Monitor (hadoop101)
Role Type: Host Monitor
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HOST_MONITOR_HOST_HEALTH Role health test bad Critical The health test result for HOST_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Server (hadoop103)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for IMPALA_IMPALADS_HEALTHY has become bad: Healthy Impala Daemon: 0. Concerning Impala Daemon: 0. Total Impala Daemon: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Health Test Results:
Health Test Name Event Code Severity Content
IMPALA_IMPALADS_HEALTHY Service health test bad Critical The health test result for IMPALA_IMPALADS_HEALTHY has become bad: Healthy Impala Daemon: 0. Concerning Impala Daemon: 0. Total Impala Daemon: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
The health test result for STATESTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Impala StateStore (hadoop103)
Role Type: Impala StateStore
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
STATESTORE_HOST_HEALTH Role health test bad Critical The health test result for STATESTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: JournalNode (hadoop103)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_HOST_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: kafka_broker (hadoop103)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HOST_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: NodeManager (hadoop102)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: JournalNode (hadoop101)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_HOST_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: kafka_broker (hadoop101)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HOST_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
The health test result for HUE_LOAD_BALANCER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Load Balancer (hadoop103)
Role Type: Load Balancer
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
HUE_LOAD_BALANCER_HOST_HEALTH Role health test bad Critical The health test result for HUE_LOAD_BALANCER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for ALERT_PUBLISHER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Alert Publisher (hadoop101)
Role Type: Alert Publisher
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
ALERT_PUBLISHER_HOST_HEALTH Role health test bad Critical The health test result for ALERT_PUBLISHER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
The health test result for SERVICE_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Service Monitor (hadoop101)
Role Type: Service Monitor
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
SERVICE_MONITOR_HOST_HEALTH Role health test bad Critical The health test result for SERVICE_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: ResourceManager (hadoop102)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Failover Controller (hadoop102)
Role Type: Failover Controller
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVERCONTROLLER_HOST_HEALTH Role health test bad Critical The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Impala Daemon (hadoop101)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_HOST_HEALTH Role health test bad Critical The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Impala Daemon (hadoop103)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_HOST_HEALTH Role health test bad Critical The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: NameNode (hadoop102)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_HOST_HEALTH Role health test bad Critical The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for CATALOGSERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Impala Catalog Server (hadoop102)
Role Type: Impala Catalog Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
CATALOGSERVER_HOST_HEALTH Role health test bad Critical The health test result for CATALOGSERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The health of the SPARK_YARN_HISTORY_SERVER is bad. The following health tests are bad: host health.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH Service health test bad Critical The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The health of the SPARK_YARN_HISTORY_SERVER is bad. The following health tests are bad: host health.
The health test result for EVENT_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
Time: Jul 18, 2024 6:15:29 PM
Monitor Startup: false
Role: Event Server (hadoop101)
Role Type: Event Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
EVENT_SERVER_HOST_HEALTH Role health test bad Critical The health test result for EVENT_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset. The following health tests are concerning: swapping.
The health test result for HUE_LOAD_BALANCER_HEALTHY has become bad: Healthy Load Balancer: 0. Concerning Load Balancer: 0. Total Load Balancer: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Time: Jul 18, 2024 6:15:34 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Health Test Results:
Health Test Name Event Code Severity Content
HUE_LOAD_BALANCER_HEALTHY Service health test bad Critical The health test result for HUE_LOAD_BALANCER_HEALTHY has become bad: Healthy Load Balancer: 0. Concerning Load Balancer: 0. Total Load Balancer: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
The health test result for ZOOKEEPER_SERVERS_HEALTHY has become bad: Healthy Server: 0. Concerning Server: 0. Total Server: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Time: Jul 18, 2024 6:15:34 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVERS_HEALTHY Service health test bad Critical The health test result for ZOOKEEPER_SERVERS_HEALTHY has become bad: Healthy Server: 0. Concerning Server: 0. Total Server: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
From noreply@localhost.localdomain Thu Jul 18 18:16:07 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 6A42842A4E82 for ; Thu, 18 Jul 2024 18:16:07 +0800 (CST) Date: Thu, 18 Jul 2024 18:16:07 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <166770808.14.1721297767434@localhost> Subject: [Cloudera Alert] 3 Alerts since 6:15:34 PM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-10008 [Cloudera Alert] 3 Alerts since 6:15:34 PM
The health test result for KAFKA_KAFKA_BROKER_HEALTHY has become bad: Healthy KAFKA_BROKER: 0. Concerning KAFKA_BROKER: 0. Total KAFKA_BROKER: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 49.99%.
Time: Jul 18, 2024 6:15:34 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HEALTHY Service health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HEALTHY has become bad: Healthy KAFKA_BROKER: 0. Concerning KAFKA_BROKER: 0. Total KAFKA_BROKER: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 49.99%.
Health test changes: 2 Became Bad
Time: Jul 18, 2024 6:15:34 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Health Test Results:
Health Test Name Event Code Severity Content
HIVE_HIVESERVER2S_HEALTHY Service health test bad Critical The health test result for HIVE_HIVESERVER2S_HEALTHY has become bad: Healthy HiveServer2: 0. Concerning HiveServer2: 0. Total HiveServer2: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
HIVE_HIVEMETASTORES_HEALTHY Service health test bad Critical The health test result for HIVE_HIVEMETASTORES_HEALTHY has become bad: Healthy Hive Metastore Server: 0. Concerning Hive Metastore Server: 0. Total Hive Metastore Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
The health test result for OOZIE_OOZIE_SERVERS_HEALTHY has become bad: Healthy Oozie Server: 0. Concerning Oozie Server: 0. Total Oozie Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Time: Jul 18, 2024 6:15:34 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: oozie
Service Display Name: Oozie
Service Type: Oozie
Health Test Results:
Health Test Name Event Code Severity Content
OOZIE_OOZIE_SERVERS_HEALTHY Service health test bad Critical The health test result for OOZIE_OOZIE_SERVERS_HEALTHY has become bad: Healthy Oozie Server: 0. Concerning Oozie Server: 0. Total Oozie Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
From noreply@localhost.localdomain Thu Jul 18 18:25:07 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 7013642A4E89 for ; Thu, 18 Jul 2024 18:25:07 +0800 (CST) Date: Thu, 18 Jul 2024 18:25:07 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1418871308.15.1721298307445@localhost> Subject: [Cloudera Alert] 3 Alerts since 6:24:41 PM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-11063 [Cloudera Alert] 3 Alerts since 6:24:41 PM
Health test changes: 2 Became Bad
Time: Jul 18, 2024 6:24:41 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Health Test Results:
Health Test Name Event Code Severity Content
IMPALA_STATESTORE_HEALTH Service health test bad Critical The health test result for IMPALA_STATESTORE_HEALTH has become bad: The Impala StateStore is not running.
IMPALA_CATALOGSERVER_HEALTH Service health test bad Critical The health test result for IMPALA_CATALOGSERVER_HEALTH has become bad: The Impala Catalog Server is not running.
The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The SPARK_YARN_HISTORY_SERVER is not running.
Time: Jul 18, 2024 6:24:47 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH Service health test bad Critical The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The SPARK_YARN_HISTORY_SERVER is not running.
The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
Time: Jul 18, 2024 6:24:52 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_JOBHISTORY_HEALTH Service health test bad Critical The health test result for YARN_JOBHISTORY_HEALTH has become bad: The JobHistory Server is not running.
From noreply@localhost.localdomain Thu Jul 18 21:57:55 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id B420C42A4141 for ; Thu, 18 Jul 2024 21:57:55 +0800 (CST) Date: Thu, 18 Jul 2024 21:57:55 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1967288700.16.1721311075719@localhost> Subject: [Cloudera Alert] 32 Alerts since 9:57:37 PM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-17568 [Cloudera Alert] 32 Alerts since 9:57:37 PM
Health test changes: 2 Became Bad
Time: Jul 18, 2024 9:57:37 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Health Test Results:
Health Test Name Event Code Severity Content
HUE_HUE_SERVERS_HEALTHY Service health test bad Critical The health test result for HUE_HUE_SERVERS_HEALTHY has become bad: Healthy Hue Server: 0. Concerning Hue Server: 0. Total Hue Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
HUE_LOAD_BALANCER_HEALTHY Service health test bad Critical The health test result for HUE_LOAD_BALANCER_HEALTHY has become bad: Healthy Load Balancer: 0. Concerning Load Balancer: 0. Total Load Balancer: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Health test changes: 3 Became Bad, 1 Became Good
Time: Jul 18, 2024 9:57:37 PM
Monitor Startup: false
Role: kafka_broker (hadoop102)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA-6.2-OFFLINE_PARTITIONS Role health test bad Critical The health test result for KAFKA-6.2-OFFLINE_PARTITIONS has become bad: Not enough data to test: This health test checks the most recent value of the kafka_offline_partitions metric and sends an alert if there are any offline partitions.
KAFKA-6.2-LAGGING_REPLICAS Role health test bad Critical The health test result for KAFKA-6.2-LAGGING_REPLICAS has become bad: Not enough data to test: This health test checks the most recent value of the kafka_under_replicated_partitions metric and sends a warning if there are any under-replicated partitions.
KAFKA-6.2-OFFLINE_DIRECTORIES Role health test bad Critical The health test result for KAFKA-6.2-OFFLINE_DIRECTORIES has become bad: Not enough data to test: This health test checks the most recent value of the kafka_logcleaner_offline_log_directory_count metric and sends a warning if there are any offline directories.
Health test changes: 1 Became Bad, 1 Became Concerning, 7 Became Good
Time: Jul 18, 2024 9:57:37 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_DATA_NODES_HEALTHY Service health test bad Critical The health test result for HDFS_DATA_NODES_HEALTHY has become bad: Healthy DataNode: 0. Concerning DataNode: 0. Total DataNode: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
Health test changes: 2 Became Bad
Time: Jul 18, 2024 9:57:37 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_CANARY_HEALTH Service health test bad Critical The health test result for ZOOKEEPER_CANARY_HEALTH has become bad: The ZooKeeper service canary failed for an unknown reason.
ZOOKEEPER_SERVERS_HEALTHY Service health test bad Critical The health test result for ZOOKEEPER_SERVERS_HEALTHY has become bad: Healthy Server: 0. Concerning Server: 0. Total Server: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
The health test result for KAFKA_KAFKA_BROKER_HEALTHY has become bad: Healthy KAFKA_BROKER: 0. Concerning KAFKA_BROKER: 0. Total KAFKA_BROKER: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 49.99%.
Time: Jul 18, 2024 9:57:37 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HEALTHY Service health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HEALTHY has become bad: Healthy KAFKA_BROKER: 0. Concerning KAFKA_BROKER: 0. Total KAFKA_BROKER: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 49.99%.
Health test changes: 1 Became Bad, 1 Became Concerning, 1 Became Good
Time: Jul 18, 2024 9:57:37 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_NODE_MANAGERS_HEALTHY Service health test bad Critical The health test result for YARN_NODE_MANAGERS_HEALTHY has become bad: Healthy NodeManager: 0. Concerning NodeManager: 0. Total NodeManager: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
Health test changes: 2 Became Bad
Time: Jul 18, 2024 9:57:37 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Health Test Results:
Health Test Name Event Code Severity Content
HIVE_HIVESERVER2S_HEALTHY Service health test bad Critical The health test result for HIVE_HIVESERVER2S_HEALTHY has become bad: Healthy HiveServer2: 0. Concerning HiveServer2: 0. Total HiveServer2: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
HIVE_HIVEMETASTORES_HEALTHY Service health test bad Critical The health test result for HIVE_HIVEMETASTORES_HEALTHY has become bad: Healthy Hive Metastore Server: 0. Concerning Hive Metastore Server: 0. Total Hive Metastore Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Health test changes: 1 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:37 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Health Test Results:
Health Test Name Event Code Severity Content
IMPALA_IMPALADS_HEALTHY Service health test bad Critical The health test result for IMPALA_IMPALADS_HEALTHY has become bad: Healthy Impala Daemon: 0. Concerning Impala Daemon: 0. Total Impala Daemon: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
The health test result for OOZIE_OOZIE_SERVERS_HEALTHY has become bad: Healthy Oozie Server: 0. Concerning Oozie Server: 0. Total Oozie Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Time: Jul 18, 2024 9:57:37 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: oozie
Service Display Name: Oozie
Service Type: Oozie
Health Test Results:
Health Test Name Event Code Severity Content
OOZIE_OOZIE_SERVERS_HEALTHY Service health test bad Critical The health test result for OOZIE_OOZIE_SERVERS_HEALTHY has become bad: Healthy Oozie Server: 0. Concerning Oozie Server: 0. Total Oozie Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Health test changes: 3 Became Bad, 1 Became Good
Time: Jul 18, 2024 9:57:37 PM
Monitor Startup: false
Role: kafka_broker (hadoop103)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA-6.2-OFFLINE_PARTITIONS Role health test bad Critical The health test result for KAFKA-6.2-OFFLINE_PARTITIONS has become bad: Not enough data to test: This health test checks the most recent value of the kafka_offline_partitions metric and sends an alert if there are any offline partitions.
KAFKA-6.2-LAGGING_REPLICAS Role health test bad Critical The health test result for KAFKA-6.2-LAGGING_REPLICAS has become bad: Not enough data to test: This health test checks the most recent value of the kafka_under_replicated_partitions metric and sends a warning if there are any under-replicated partitions.
KAFKA-6.2-OFFLINE_DIRECTORIES Role health test bad Critical The health test result for KAFKA-6.2-OFFLINE_DIRECTORIES has become bad: Not enough data to test: This health test checks the most recent value of the kafka_logcleaner_offline_log_directory_count metric and sends a warning if there are any offline directories.
Health test changes: 3 Became Bad, 1 Became Good
Time: Jul 18, 2024 9:57:37 PM
Monitor Startup: false
Role: kafka_broker (hadoop101)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA-6.2-OFFLINE_PARTITIONS Role health test bad Critical The health test result for KAFKA-6.2-OFFLINE_PARTITIONS has become bad: Not enough data to test: This health test checks the most recent value of the kafka_offline_partitions metric and sends an alert if there are any offline partitions.
KAFKA-6.2-LAGGING_REPLICAS Role health test bad Critical The health test result for KAFKA-6.2-LAGGING_REPLICAS has become bad: Not enough data to test: This health test checks the most recent value of the kafka_under_replicated_partitions metric and sends a warning if there are any under-replicated partitions.
KAFKA-6.2-OFFLINE_DIRECTORIES Role health test bad Critical The health test result for KAFKA-6.2-OFFLINE_DIRECTORIES has become bad: Not enough data to test: This health test checks the most recent value of the kafka_logcleaner_offline_log_directory_count metric and sends a warning if there are any offline directories.
Health test changes: 1 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Server (hadoop102)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HIVEMETASTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Hive Metastore Server (hadoop101)
Role Type: Hive Metastore Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HIVEMETASTORE_HOST_HEALTH Role health test bad Critical The health test result for HIVEMETASTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Hue Server (hadoop102)
Role Type: Hue Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
HUE_SERVER_HOST_HEALTH Role health test bad Critical The health test result for HUE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
HUE_SERVER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for HUE_SERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: NodeManager (hadoop101)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
NODE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NODE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 5 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: kafka_broker (hadoop102)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HOST_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Health test changes: 2 Became Bad, 1 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_HA_NAMENODE_HEALTH Service health test bad Critical The health test result for HDFS_HA_NAMENODE_HEALTH has become bad: NameNode summary: Hadoop101 (Availability: Active, Health: Bad), Hadoop102 (Availability: Standby, Health: Bad). This health test reflects the health of the active NameNode.
HDFS_FAILOVER_CONTROLLERS_HEALTHY Service health test bad Critical The health test result for HDFS_FAILOVER_CONTROLLERS_HEALTHY has become bad: Failover Controllers with bad health: Hadoop102, Hadoop101. The following health tests are bad: host health. The following health tests are bad: host health.
Health test changes: 1 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Failover Controller (hadoop101)
Role Type: Failover Controller
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVERCONTROLLER_HOST_HEALTH Role health test bad Critical The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Health test changes: 1 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Oozie Server (hadoop102)
Role Type: Oozie Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: oozie
Service Display Name: Oozie
Service Type: Oozie
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
OOZIE_SERVER_HOST_HEALTH Role health test bad Critical The health test result for OOZIE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Health test changes: 1 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: spark_yarn_history_server (hadoop103)
Role Type: SPARK_YARN_HISTORY_SERVER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HOST_HEALTH Role health test bad Critical The health test result for SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: JournalNode (hadoop102)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_HOST_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
JOURNAL_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for JOURNAL_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for HIVESERVER2_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: HiveServer2 (hadoop101)
Role Type: HiveServer2
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HIVESERVER2_HOST_HEALTH Role health test bad Critical The health test result for HIVESERVER2_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: NodeManager (hadoop103)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
NODE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NODE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: ResourceManager (hadoop103)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
RESOURCE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for RESOURCE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: NameNode (hadoop101)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_HOST_HEALTH Role health test bad Critical The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
NAME_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NAME_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Impala Daemon (hadoop102)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_HOST_HEALTH Role health test bad Critical The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
IMPALAD_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for IMPALAD_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 1 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_RESOURCEMANAGERS_HEALTH Service health test bad Critical The health test result for YARN_RESOURCEMANAGERS_HEALTH has become bad: ResourceManager summary: Hadoop102 (Availability: Active, Health: Bad), Hadoop103 (Availability: Unknown, Health: Bad). This health test reflects the health of the active ResourceManager.
YARN_JOBHISTORY_HEALTH Service health test bad Critical The health test result for YARN_JOBHISTORY_HEALTH has become bad: The health of the JobHistory Server is bad. The following health tests are bad: host health, web server status.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: JobHistory Server (hadoop103)
Role Type: JobHistory Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
JOBHISTORY_HOST_HEALTH Role health test bad Critical The health test result for JOBHISTORY_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
JOBHISTORY_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for JOBHISTORY_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Server (hadoop101)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Host Monitor (hadoop101)
Role Type: Host Monitor
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HOST_MONITOR_HOST_HEALTH Role health test bad Critical The health test result for HOST_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
HOST_MONITOR_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for HOST_MONITOR_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Server (hadoop103)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Health test changes: 2 Became Bad, 1 Became Concerning
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Health Test Results:
Health Test Name Event Code Severity Content
IMPALA_STATESTORE_HEALTH Service health test bad Critical The health test result for IMPALA_STATESTORE_HEALTH has become bad: The health of the Impala StateStore is bad. The following health tests are bad: host health, web server status.
IMPALA_CATALOGSERVER_HEALTH Service health test bad Critical The health test result for IMPALA_CATALOGSERVER_HEALTH has become bad: The health of the Impala Catalog Server is bad. The following health tests are bad: web server status, host health.
From noreply@localhost.localdomain Thu Jul 18 21:58:38 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id A43F542A414E for ; Thu, 18 Jul 2024 21:58:38 +0800 (CST) Date: Thu, 18 Jul 2024 21:58:38 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1071091298.17.1721311118670@localhost> Subject: [Cloudera Alert] 25 Alerts since 9:57:42 PM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-18195 [Cloudera Alert] 25 Alerts since 9:57:42 PM
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Impala StateStore (hadoop103)
Role Type: Impala StateStore
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
STATESTORE_HOST_HEALTH Role health test bad Critical The health test result for STATESTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
STATESTORE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for STATESTORE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: JournalNode (hadoop103)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_HOST_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
JOURNAL_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for JOURNAL_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 5 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: kafka_broker (hadoop103)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HOST_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: NodeManager (hadoop102)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
NODE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NODE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: JournalNode (hadoop101)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_HOST_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
JOURNAL_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for JOURNAL_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 5 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: kafka_broker (hadoop101)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HOST_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Health test changes: 1 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Load Balancer (hadoop103)
Role Type: Load Balancer
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
HUE_LOAD_BALANCER_HOST_HEALTH Role health test bad Critical The health test result for HUE_LOAD_BALANCER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Health test changes: 1 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Alert Publisher (hadoop101)
Role Type: Alert Publisher
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
ALERT_PUBLISHER_HOST_HEALTH Role health test bad Critical The health test result for ALERT_PUBLISHER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Service Monitor (hadoop101)
Role Type: Service Monitor
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
SERVICE_MONITOR_HOST_HEALTH Role health test bad Critical The health test result for SERVICE_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
SERVICE_MONITOR_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for SERVICE_MONITOR_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: ResourceManager (hadoop102)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
RESOURCE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for RESOURCE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Failover Controller (hadoop102)
Role Type: Failover Controller
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVERCONTROLLER_HOST_HEALTH Role health test bad Critical The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Impala Daemon (hadoop101)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_HOST_HEALTH Role health test bad Critical The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
IMPALAD_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for IMPALAD_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Impala Daemon (hadoop103)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_HOST_HEALTH Role health test bad Critical The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
IMPALAD_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for IMPALAD_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: NameNode (hadoop102)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_HOST_HEALTH Role health test bad Critical The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
NAME_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NAME_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Impala Catalog Server (hadoop102)
Role Type: Impala Catalog Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
CATALOGSERVER_HOST_HEALTH Role health test bad Critical The health test result for CATALOGSERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
CATALOGSERVER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for CATALOGSERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The health of the SPARK_YARN_HISTORY_SERVER is bad. The following health tests are bad: host health.
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH Service health test bad Critical The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The health of the SPARK_YARN_HISTORY_SERVER is bad. The following health tests are bad: host health.
Health test changes: 2 Became Bad, 2 Became Good
Time: Jul 18, 2024 9:57:42 PM
Monitor Startup: false
Role: Event Server (hadoop101)
Role Type: Event Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
EVENT_SERVER_HOST_HEALTH Role health test bad Critical The health test result for EVENT_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
EVENT_SERVER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for EVENT_SERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad
Time: Jul 18, 2024 9:57:47 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Health Test Results:
Health Test Name Event Code Severity Content
HUE_HUE_SERVERS_HEALTHY Service health test bad Critical The health test result for HUE_HUE_SERVERS_HEALTHY has become bad: Healthy Hue Server: 0. Concerning Hue Server: 0. Total Hue Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
HUE_LOAD_BALANCER_HEALTHY Service health test bad Critical The health test result for HUE_LOAD_BALANCER_HEALTHY has become bad: Healthy Load Balancer: 0. Concerning Load Balancer: 0. Total Load Balancer: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
The health test result for HDFS_DATA_NODES_HEALTHY has become bad: Healthy DataNode: 0. Concerning DataNode: 0. Total DataNode: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
Time: Jul 18, 2024 9:57:47 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_DATA_NODES_HEALTHY Service health test bad Critical The health test result for HDFS_DATA_NODES_HEALTHY has become bad: Healthy DataNode: 0. Concerning DataNode: 0. Total DataNode: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
The health test result for ZOOKEEPER_SERVERS_HEALTHY has become bad: Healthy Server: 0. Concerning Server: 0. Total Server: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Time: Jul 18, 2024 9:57:47 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVERS_HEALTHY Service health test bad Critical The health test result for ZOOKEEPER_SERVERS_HEALTHY has become bad: Healthy Server: 0. Concerning Server: 0. Total Server: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
The health test result for YARN_NODE_MANAGERS_HEALTHY has become bad: Healthy NodeManager: 0. Concerning NodeManager: 0. Total NodeManager: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
Time: Jul 18, 2024 9:57:47 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_NODE_MANAGERS_HEALTHY Service health test bad Critical The health test result for YARN_NODE_MANAGERS_HEALTHY has become bad: Healthy NodeManager: 0. Concerning NodeManager: 0. Total NodeManager: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
Health test changes: 2 Became Bad
Time: Jul 18, 2024 9:57:47 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Health Test Results:
Health Test Name Event Code Severity Content
HIVE_HIVESERVER2S_HEALTHY Service health test bad Critical The health test result for HIVE_HIVESERVER2S_HEALTHY has become bad: Healthy HiveServer2: 0. Concerning HiveServer2: 0. Total HiveServer2: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
HIVE_HIVEMETASTORES_HEALTHY Service health test bad Critical The health test result for HIVE_HIVEMETASTORES_HEALTHY has become bad: Healthy Hive Metastore Server: 0. Concerning Hive Metastore Server: 0. Total Hive Metastore Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
The health test result for IMPALA_IMPALADS_HEALTHY has become bad: Healthy Impala Daemon: 0. Concerning Impala Daemon: 0. Total Impala Daemon: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
Time: Jul 18, 2024 9:57:47 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Health Test Results:
Health Test Name Event Code Severity Content
IMPALA_IMPALADS_HEALTHY Service health test bad Critical The health test result for IMPALA_IMPALADS_HEALTHY has become bad: Healthy Impala Daemon: 0. Concerning Impala Daemon: 0. Total Impala Daemon: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
The health test result for OOZIE_OOZIE_SERVERS_HEALTHY has become bad: Healthy Oozie Server: 0. Concerning Oozie Server: 0. Total Oozie Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Time: Jul 18, 2024 9:57:47 PM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: oozie
Service Display Name: Oozie
Service Type: Oozie
Health Test Results:
Health Test Name Event Code Severity Content
OOZIE_OOZIE_SERVERS_HEALTHY Service health test bad Critical The health test result for OOZIE_OOZIE_SERVERS_HEALTHY has become bad: Healthy Oozie Server: 0. Concerning Oozie Server: 0. Total Oozie Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Health test changes: 1 Became Bad, 4 Became Good
Time: Jul 18, 2024 9:57:52 PM
Monitor Startup: false
Role: Server (hadoop103)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_QUORUM_MEMBERSHIP Role health test bad Critical The health test result for ZOOKEEPER_SERVER_QUORUM_MEMBERSHIP has become bad: Quorum membership status could not be detected for the last 3 minute(s). The last connection attempt to the ZooKeeper server to determine the quorum membership status failed.
From noreply@localhost.localdomain Thu Aug 8 09:56:03 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 5380D42A5FA9 for ; Thu, 8 Aug 2024 09:56:03 +0800 (CST) Date: Thu, 8 Aug 2024 09:56:03 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1870843609.18.1723082163314@localhost> Subject: [Cloudera Alert] 32 Alerts since 9:55:53 AM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-21647 [Cloudera Alert] 32 Alerts since 9:55:53 AM
Health test changes: 2 Became Bad
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Health Test Results:
Health Test Name Event Code Severity Content
HUE_HUE_SERVERS_HEALTHY Service health test bad Critical The health test result for HUE_HUE_SERVERS_HEALTHY has become bad: Healthy Hue Server: 0. Concerning Hue Server: 0. Total Hue Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
HUE_LOAD_BALANCER_HEALTHY Service health test bad Critical The health test result for HUE_LOAD_BALANCER_HEALTHY has become bad: Healthy Load Balancer: 0. Concerning Load Balancer: 0. Total Load Balancer: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Health test changes: 1 Became Bad, 4 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: NodeManager (hadoop101)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NODE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 3 Became Bad, 1 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: kafka_broker (hadoop102)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA-6.2-OFFLINE_PARTITIONS Role health test bad Critical The health test result for KAFKA-6.2-OFFLINE_PARTITIONS has become bad: Not enough data to test: This health test checks the most recent value of the kafka_offline_partitions metric and sends an alert if there are any offline partitions.
KAFKA-6.2-LAGGING_REPLICAS Role health test bad Critical The health test result for KAFKA-6.2-LAGGING_REPLICAS has become bad: Not enough data to test: This health test checks the most recent value of the kafka_under_replicated_partitions metric and sends a warning if there are any under-replicated partitions.
KAFKA-6.2-OFFLINE_DIRECTORIES Role health test bad Critical The health test result for KAFKA-6.2-OFFLINE_DIRECTORIES has become bad: Not enough data to test: This health test checks the most recent value of the kafka_logcleaner_offline_log_directory_count metric and sends a warning if there are any offline directories.
Health test changes: 2 Became Bad, 1 Became Concerning, 6 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_HA_NAMENODE_HEALTH Service health test bad Critical The health test result for HDFS_HA_NAMENODE_HEALTH has become bad: NameNode summary: Hadoop101 (Availability: Active, Health: Bad), Hadoop102 (Availability: Unknown, Health: Good). This health test reflects the health of the active NameNode.
HDFS_DATA_NODES_HEALTHY Service health test bad Critical The health test result for HDFS_DATA_NODES_HEALTHY has become bad: Healthy DataNode: 0. Concerning DataNode: 0. Total DataNode: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
Health test changes: 1 Became Bad, 1 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: Oozie Server (hadoop102)
Role Type: Oozie Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: oozie
Service Display Name: Oozie
Service Type: Oozie
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
OOZIE_SERVER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for OOZIE_SERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_CANARY_HEALTH Service health test bad Critical The health test result for ZOOKEEPER_CANARY_HEALTH has become bad: The ZooKeeper service canary failed for an unknown reason.
ZOOKEEPER_SERVERS_HEALTHY Service health test bad Critical The health test result for ZOOKEEPER_SERVERS_HEALTHY has become bad: Healthy Server: 0. Concerning Server: 0. Total Server: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Health test changes: 1 Became Bad, 4 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: NodeManager (hadoop103)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NODE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 3 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: ResourceManager (hadoop103)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for RESOURCE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 5 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: NameNode (hadoop101)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_HA_CHECKPOINT_AGE Role health test bad Critical The health test result for NAME_NODE_HA_CHECKPOINT_AGE has become bad: The filesystem checkpoint is 20 day(s), 11 hour(s), 18 minute(s) old. This is 49,131.58% of the configured checkpoint period of 1 hour(s). Critical threshold: 400.00%. 165 transactions have occurred since the last filesystem checkpoint. This is 0.02% of the configured checkpoint transaction target of 1,000,000.
NAME_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NAME_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for KAFKA_KAFKA_BROKER_HEALTHY has become bad: Healthy KAFKA_BROKER: 0. Concerning KAFKA_BROKER: 0. Total KAFKA_BROKER: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 49.99%.
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HEALTHY Service health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HEALTHY has become bad: Healthy KAFKA_BROKER: 0. Concerning KAFKA_BROKER: 0. Total KAFKA_BROKER: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 49.99%.
Health test changes: 3 Became Bad
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_RESOURCEMANAGERS_HEALTH Service health test bad Critical The health test result for YARN_RESOURCEMANAGERS_HEALTH has become bad: ResourceManager summary: Hadoop102 (Availability: Unknown, Health: Good), Hadoop103 (Availability: Unknown, Health: Bad). This health test is bad because the Service Monitor did not find an active ResourceManager.
YARN_JOBHISTORY_HEALTH Service health test bad Critical The health test result for YARN_JOBHISTORY_HEALTH has become bad: The health of the JobHistory Server is bad. The following health tests are bad: web server status.
YARN_NODE_MANAGERS_HEALTHY Service health test bad Critical The health test result for YARN_NODE_MANAGERS_HEALTHY has become bad: Healthy NodeManager: 0. Concerning NodeManager: 0. Total NodeManager: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
Health test changes: 1 Became Bad, 3 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: JobHistory Server (hadoop103)
Role Type: JobHistory Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
JOBHISTORY_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for JOBHISTORY_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Health Test Results:
Health Test Name Event Code Severity Content
HIVE_HIVESERVER2S_HEALTHY Service health test bad Critical The health test result for HIVE_HIVESERVER2S_HEALTHY has become bad: Healthy HiveServer2: 0. Concerning HiveServer2: 0. Total HiveServer2: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
HIVE_HIVEMETASTORES_HEALTHY Service health test bad Critical The health test result for HIVE_HIVEMETASTORES_HEALTHY has become bad: Healthy Hive Metastore Server: 0. Concerning Hive Metastore Server: 0. Total Hive Metastore Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Health test changes: 1 Became Bad, 3 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: Host Monitor (hadoop101)
Role Type: Host Monitor
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HOST_MONITOR_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for HOST_MONITOR_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 2 Became Bad, 1 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Health Test Results:
Health Test Name Event Code Severity Content
IMPALA_STATESTORE_HEALTH Service health test bad Critical The health test result for IMPALA_STATESTORE_HEALTH has become bad: The health of the Impala StateStore is bad. The following health tests are bad: web server status.
IMPALA_IMPALADS_HEALTHY Service health test bad Critical The health test result for IMPALA_IMPALADS_HEALTHY has become bad: Healthy Impala Daemon: 0. Concerning Impala Daemon: 0. Total Impala Daemon: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 90.00%.
Health test changes: 1 Became Bad, 3 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: Impala StateStore (hadoop103)
Role Type: Impala StateStore
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
STATESTORE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for STATESTORE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for OOZIE_OOZIE_SERVERS_HEALTHY has become bad: Healthy Oozie Server: 0. Concerning Oozie Server: 0. Total Oozie Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: oozie
Service Display Name: Oozie
Service Type: Oozie
Health Test Results:
Health Test Name Event Code Severity Content
OOZIE_OOZIE_SERVERS_HEALTHY Service health test bad Critical The health test result for OOZIE_OOZIE_SERVERS_HEALTHY has become bad: Healthy Oozie Server: 0. Concerning Oozie Server: 0. Total Oozie Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Health test changes: 1 Became Bad, 4 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: JournalNode (hadoop103)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for JOURNAL_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 4 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: JournalNode (hadoop101)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for JOURNAL_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 3 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: Service Monitor (hadoop101)
Role Type: Service Monitor
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
SERVICE_MONITOR_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for SERVICE_MONITOR_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 1 Became Concerning, 3 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: Impala Daemon (hadoop101)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for IMPALAD_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 3 Became Good
Time: Aug 8, 2024 9:55:53 AM
Monitor Startup: false
Role: Event Server (hadoop101)
Role Type: Event Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
EVENT_SERVER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for EVENT_SERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Server (hadoop102)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
The health test result for HIVEMETASTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Hive Metastore Server (hadoop101)
Role Type: Hive Metastore Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HIVEMETASTORE_HOST_HEALTH Role health test bad Critical The health test result for HIVEMETASTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HUE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Hue Server (hadoop102)
Role Type: Hue Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
HUE_SERVER_HOST_HEALTH Role health test bad Critical The health test result for HUE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: NodeManager (hadoop101)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: kafka_broker (hadoop102)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HOST_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
The health test result for HDFS_FAILOVER_CONTROLLERS_HEALTHY has become bad: Failover Controllers with bad health: Hadoop102, Hadoop101. The following health tests are bad: host health. The following health tests are bad: host health.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVER_CONTROLLERS_HEALTHY Service health test bad Critical The health test result for HDFS_FAILOVER_CONTROLLERS_HEALTHY has become bad: Failover Controllers with bad health: Hadoop102, Hadoop101. The following health tests are bad: host health. The following health tests are bad: host health.
The health test result for OOZIE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Oozie Server (hadoop102)
Role Type: Oozie Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: oozie
Service Display Name: Oozie
Service Type: Oozie
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
OOZIE_SERVER_HOST_HEALTH Role health test bad Critical The health test result for OOZIE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Failover Controller (hadoop101)
Role Type: Failover Controller
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVERCONTROLLER_HOST_HEALTH Role health test bad Critical The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: spark_yarn_history_server (hadoop103)
Role Type: SPARK_YARN_HISTORY_SERVER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HOST_HEALTH Role health test bad Critical The health test result for SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: JournalNode (hadoop102)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_HOST_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
From noreply@localhost.localdomain Thu Aug 8 09:56:15 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 5516042A5FA9 for ; Thu, 8 Aug 2024 09:56:15 +0800 (CST) Date: Thu, 8 Aug 2024 09:56:15 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <810631359.19.1723082175347@localhost> Subject: [Cloudera Alert] 31 Alerts since 9:55:58 AM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-21835 [Cloudera Alert] 31 Alerts since 9:55:58 AM
The health test result for HIVESERVER2_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: HiveServer2 (hadoop101)
Role Type: HiveServer2
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HIVESERVER2_HOST_HEALTH Role health test bad Critical The health test result for HIVESERVER2_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: NodeManager (hadoop103)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: ResourceManager (hadoop103)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: NameNode (hadoop101)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_HOST_HEALTH Role health test bad Critical The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Impala Daemon (hadoop102)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_HOST_HEALTH Role health test bad Critical The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
The health test result for JOBHISTORY_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: JobHistory Server (hadoop103)
Role Type: JobHistory Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
JOBHISTORY_HOST_HEALTH Role health test bad Critical The health test result for JOBHISTORY_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Server (hadoop101)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Server (hadoop103)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HOST_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Host Monitor (hadoop101)
Role Type: Host Monitor
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HOST_MONITOR_HOST_HEALTH Role health test bad Critical The health test result for HOST_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for IMPALA_CATALOGSERVER_HEALTH has become bad: The health of the Impala Catalog Server is bad. The following health tests are bad: host health.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Health Test Results:
Health Test Name Event Code Severity Content
IMPALA_CATALOGSERVER_HEALTH Service health test bad Critical The health test result for IMPALA_CATALOGSERVER_HEALTH has become bad: The health of the Impala Catalog Server is bad. The following health tests are bad: host health.
The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: JournalNode (hadoop103)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_HOST_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for STATESTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Impala StateStore (hadoop103)
Role Type: Impala StateStore
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
STATESTORE_HOST_HEALTH Role health test bad Critical The health test result for STATESTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: NodeManager (hadoop102)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: kafka_broker (hadoop103)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HOST_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: JournalNode (hadoop101)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_HOST_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: kafka_broker (hadoop101)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HOST_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for ALERT_PUBLISHER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Alert Publisher (hadoop101)
Role Type: Alert Publisher
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
ALERT_PUBLISHER_HOST_HEALTH Role health test bad Critical The health test result for ALERT_PUBLISHER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HUE_LOAD_BALANCER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Load Balancer (hadoop103)
Role Type: Load Balancer
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
HUE_LOAD_BALANCER_HOST_HEALTH Role health test bad Critical The health test result for HUE_LOAD_BALANCER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for SERVICE_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Service Monitor (hadoop101)
Role Type: Service Monitor
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
SERVICE_MONITOR_HOST_HEALTH Role health test bad Critical The health test result for SERVICE_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: ResourceManager (hadoop102)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Failover Controller (hadoop102)
Role Type: Failover Controller
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVERCONTROLLER_HOST_HEALTH Role health test bad Critical The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Impala Daemon (hadoop101)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_HOST_HEALTH Role health test bad Critical The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Impala Daemon (hadoop103)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_HOST_HEALTH Role health test bad Critical The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: NameNode (hadoop102)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_HOST_HEALTH Role health test bad Critical The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
The health test result for CATALOGSERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Impala Catalog Server (hadoop102)
Role Type: Impala Catalog Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
CATALOGSERVER_HOST_HEALTH Role health test bad Critical The health test result for CATALOGSERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: agent status.
The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The health of the SPARK_YARN_HISTORY_SERVER is bad. The following health tests are bad: host health.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH Service health test bad Critical The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The health of the SPARK_YARN_HISTORY_SERVER is bad. The following health tests are bad: host health.
The health test result for EVENT_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 9:55:58 AM
Monitor Startup: false
Role: Event Server (hadoop101)
Role Type: Event Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
EVENT_SERVER_HOST_HEALTH Role health test bad Critical The health test result for EVENT_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Health test changes: 2 Became Bad
Time: Aug 8, 2024 9:56:03 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Health Test Results:
Health Test Name Event Code Severity Content
HUE_HUE_SERVERS_HEALTHY Service health test bad Critical The health test result for HUE_HUE_SERVERS_HEALTHY has become bad: Healthy Hue Server: 0. Concerning Hue Server: 0. Total Hue Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
HUE_LOAD_BALANCER_HEALTHY Service health test bad Critical The health test result for HUE_LOAD_BALANCER_HEALTHY has become bad: Healthy Load Balancer: 0. Concerning Load Balancer: 0. Total Load Balancer: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
The health test result for ZOOKEEPER_SERVERS_HEALTHY has become bad: Healthy Server: 0. Concerning Server: 0. Total Server: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Time: Aug 8, 2024 9:56:03 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVERS_HEALTHY Service health test bad Critical The health test result for ZOOKEEPER_SERVERS_HEALTHY has become bad: Healthy Server: 0. Concerning Server: 0. Total Server: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
The health test result for KAFKA_KAFKA_BROKER_HEALTHY has become bad: Healthy KAFKA_BROKER: 0. Concerning KAFKA_BROKER: 0. Total KAFKA_BROKER: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 49.99%.
Time: Aug 8, 2024 9:56:03 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HEALTHY Service health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HEALTHY has become bad: Healthy KAFKA_BROKER: 0. Concerning KAFKA_BROKER: 0. Total KAFKA_BROKER: 3. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 49.99%.
Health test changes: 2 Became Bad
Time: Aug 8, 2024 9:56:03 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Health Test Results:
Health Test Name Event Code Severity Content
HIVE_HIVESERVER2S_HEALTHY Service health test bad Critical The health test result for HIVE_HIVESERVER2S_HEALTHY has become bad: Healthy HiveServer2: 0. Concerning HiveServer2: 0. Total HiveServer2: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
HIVE_HIVEMETASTORES_HEALTHY Service health test bad Critical The health test result for HIVE_HIVEMETASTORES_HEALTHY has become bad: Healthy Hive Metastore Server: 0. Concerning Hive Metastore Server: 0. Total Hive Metastore Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
From noreply@localhost.localdomain Thu Aug 8 09:57:15 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 5914F42A5FAA for ; Thu, 8 Aug 2024 09:57:15 +0800 (CST) Date: Thu, 8 Aug 2024 09:57:15 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <92092884.20.1723082235354@localhost> Subject: [Cloudera Alert] 20 Alerts since 9:56:18 AM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-22392 [Cloudera Alert] 20 Alerts since 9:56:18 AM
The health test result for ZOOKEEPER_SERVER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
Time: Aug 8, 2024 9:56:18 AM
Monitor Startup: false
Role: Server (hadoop102)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_SCM_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
The health test result for HUE_SERVER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
Time: Aug 8, 2024 9:56:18 AM
Monitor Startup: false
Role: Hue Server (hadoop102)
Role Type: Hue Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
HUE_SERVER_SCM_HEALTH Role health test bad Critical The health test result for HUE_SERVER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
The health test result for KAFKA_KAFKA_BROKER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
Time: Aug 8, 2024 9:56:18 AM
Monitor Startup: false
Role: kafka_broker (hadoop102)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_SCM_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
The health test result for OOZIE_SERVER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
Time: Aug 8, 2024 9:56:18 AM
Monitor Startup: false
Role: Oozie Server (hadoop102)
Role Type: Oozie Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: oozie
Service Display Name: Oozie
Service Type: Oozie
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
OOZIE_SERVER_SCM_HEALTH Role health test bad Critical The health test result for OOZIE_SERVER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
The health test result for JOURNAL_NODE_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
Time: Aug 8, 2024 9:56:18 AM
Monitor Startup: false
Role: JournalNode (hadoop102)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_SCM_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
The health test result for IMPALAD_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
Time: Aug 8, 2024 9:56:18 AM
Monitor Startup: false
Role: Impala Daemon (hadoop102)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_SCM_HEALTH Role health test bad Critical The health test result for IMPALAD_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
The health test result for NODE_MANAGER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
Time: Aug 8, 2024 9:56:18 AM
Monitor Startup: false
Role: NodeManager (hadoop102)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_SCM_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
The health test result for RESOURCE_MANAGER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
Time: Aug 8, 2024 9:56:18 AM
Monitor Startup: false
Role: ResourceManager (hadoop102)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_SCM_HEALTH Role health test bad Critical The health test result for RESOURCE_MANAGER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
The health test result for HDFS_FAILOVERCONTROLLER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
Time: Aug 8, 2024 9:56:18 AM
Monitor Startup: false
Role: Failover Controller (hadoop102)
Role Type: Failover Controller
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVERCONTROLLER_SCM_HEALTH Role health test bad Critical The health test result for HDFS_FAILOVERCONTROLLER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
The health test result for NAME_NODE_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
Time: Aug 8, 2024 9:56:18 AM
Monitor Startup: false
Role: NameNode (hadoop102)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_SCM_HEALTH Role health test bad Critical The health test result for NAME_NODE_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
The health test result for CATALOGSERVER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
Time: Aug 8, 2024 9:56:18 AM
Monitor Startup: false
Role: Impala Catalog Server (hadoop102)
Role Type: Impala Catalog Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
CATALOGSERVER_SCM_HEALTH Role health test bad Critical The health test result for CATALOGSERVER_SCM_HEALTH has become bad: This role's host has been out of contact with Cloudera Manager for too long.
The health test result for ZOOKEEPER_SERVER_QUORUM_MEMBERSHIP has become bad: Quorum membership status could not be detected for the last 3 minute(s). The last connection attempt to the ZooKeeper server to determine the quorum membership status failed.
Time: Aug 8, 2024 9:56:28 AM
Monitor Startup: false
Role: Server (hadoop102)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_QUORUM_MEMBERSHIP Role health test bad Critical The health test result for ZOOKEEPER_SERVER_QUORUM_MEMBERSHIP has become bad: Quorum membership status could not be detected for the last 3 minute(s). The last connection attempt to the ZooKeeper server to determine the quorum membership status failed.
Health test changes: 1 Became Bad, 5 Became Good
Time: Aug 8, 2024 9:56:28 AM
Monitor Startup: false
Role: Server (hadoop103)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_QUORUM_MEMBERSHIP Role health test bad Critical The health test result for ZOOKEEPER_SERVER_QUORUM_MEMBERSHIP has become bad: Quorum membership status could not be detected for the last 3 minute(s). The last connection attempt to the ZooKeeper server to determine the quorum membership status failed.
The health test result for NODE_MANAGER_CONNECTIVITY has become bad: This NodeManager is not connected to its ResourceManager.
Time: Aug 8, 2024 9:56:53 AM
Monitor Startup: false
Role: NodeManager (hadoop101)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_CONNECTIVITY Role health test bad Critical The health test result for NODE_MANAGER_CONNECTIVITY has become bad: This NodeManager is not connected to its ResourceManager.
The health test result for JOURNAL_NODE_SYNC_STATUS has become bad: The active NameNode is out of sync with this JournalNode.
Time: Aug 8, 2024 9:56:53 AM
Monitor Startup: false
Role: JournalNode (hadoop102)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_SYNC_STATUS Role health test bad Critical The health test result for JOURNAL_NODE_SYNC_STATUS has become bad: The active NameNode is out of sync with this JournalNode.
The health test result for NODE_MANAGER_CONNECTIVITY has become bad: This NodeManager is not connected to its ResourceManager.
Time: Aug 8, 2024 9:56:53 AM
Monitor Startup: false
Role: NodeManager (hadoop103)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_CONNECTIVITY Role health test bad Critical The health test result for NODE_MANAGER_CONNECTIVITY has become bad: This NodeManager is not connected to its ResourceManager.
The health test result for NAME_NODE_JOURNAL_NODE_SYNC_STATUS has become bad: JournalNodes out of sync: Hadoop102. JournalNodes in sync: Hadoop101, Hadoop103.
Time: Aug 8, 2024 9:56:53 AM
Monitor Startup: false
Role: NameNode (hadoop101)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_JOURNAL_NODE_SYNC_STATUS Role health test bad Critical The health test result for NAME_NODE_JOURNAL_NODE_SYNC_STATUS has become bad: JournalNodes out of sync: Hadoop102. JournalNodes in sync: Hadoop101, Hadoop103.
The health test result for IMPALAD_CONNECTIVITY has become bad: This Impala Daemon is not connected to its StateStore.
Time: Aug 8, 2024 9:56:53 AM
Monitor Startup: false
Role: Impala Daemon (hadoop102)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_CONNECTIVITY Role health test bad Critical The health test result for IMPALAD_CONNECTIVITY has become bad: This Impala Daemon is not connected to its StateStore.
The health test result for NODE_MANAGER_CONNECTIVITY has become bad: This NodeManager is not connected to its ResourceManager.
Time: Aug 8, 2024 9:56:53 AM
Monitor Startup: false
Role: NodeManager (hadoop102)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_CONNECTIVITY Role health test bad Critical The health test result for NODE_MANAGER_CONNECTIVITY has become bad: This NodeManager is not connected to its ResourceManager.
The health test result for CATALOGSERVER_CONNECTIVITY has become bad: This Catalog Server is not connected to its StateStore.
Time: Aug 8, 2024 9:56:53 AM
Monitor Startup: false
Role: Impala Catalog Server (hadoop102)
Role Type: Impala Catalog Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
CATALOGSERVER_CONNECTIVITY Role health test bad Critical The health test result for CATALOGSERVER_CONNECTIVITY has become bad: This Catalog Server is not connected to its StateStore.
From noreply@localhost.localdomain Thu Aug 8 09:58:15 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 5C6D342A42A6 for ; Thu, 8 Aug 2024 09:58:15 +0800 (CST) Date: Thu, 8 Aug 2024 09:58:15 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1118251126.21.1723082295366@localhost> Subject: [Cloudera Alert] 6 Alerts since 9:57:08 AM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-22967 [Cloudera Alert] 6 Alerts since 9:57:08 AM
Health test changes: 1 Became Bad, 5 Became Good
Time: Aug 8, 2024 9:57:08 AM
Monitor Startup: false
Role: JournalNode (hadoop102)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for JOURNAL_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 5 Became Good
Time: Aug 8, 2024 9:57:08 AM
Monitor Startup: false
Role: Impala Daemon (hadoop102)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for IMPALAD_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 7 Became Good
Time: Aug 8, 2024 9:57:08 AM
Monitor Startup: false
Role: NodeManager (hadoop102)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NODE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 4 Became Good
Time: Aug 8, 2024 9:57:08 AM
Monitor Startup: false
Role: ResourceManager (hadoop102)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for RESOURCE_MANAGER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 5 Became Good
Time: Aug 8, 2024 9:57:08 AM
Monitor Startup: false
Role: NameNode (hadoop102)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for NAME_NODE_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
Health test changes: 1 Became Bad, 4 Became Good
Time: Aug 8, 2024 9:57:08 AM
Monitor Startup: false
Role: Impala Catalog Server (hadoop102)
Role Type: Impala Catalog Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
CATALOGSERVER_WEB_METRIC_COLLECTION Role health test bad Critical The health test result for CATALOGSERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server.
From noreply@localhost.localdomain Thu Aug 8 10:02:35 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id EC3ED42A4151 for ; Thu, 8 Aug 2024 10:02:34 +0800 (CST) Date: Thu, 8 Aug 2024 10:02:34 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1861900325.22.1723082554953@localhost> Subject: [Cloudera Alert] 32 Alerts since 10:02:10 AM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-23954 [Cloudera Alert] 32 Alerts since 10:02:10 AM
The health test result for SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Role: spark_yarn_history_server (hadoop103)
Role Type: SPARK_YARN_HISTORY_SERVER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HOST_HEALTH Role health test bad Critical The health test result for SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Role: NodeManager (hadoop103)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Role: ResourceManager (hadoop103)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Health test changes: 2 Became Bad
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_RESOURCEMANAGERS_HEALTH Service health test bad Critical The health test result for YARN_RESOURCEMANAGERS_HEALTH has become bad: ResourceManager summary: Hadoop103 (Availability: Active, Health: Bad), Hadoop102 (Availability: Standby, Health: Good). This health test reflects the health of the active ResourceManager.
YARN_JOBHISTORY_HEALTH Service health test bad Critical The health test result for YARN_JOBHISTORY_HEALTH has become bad: The health of the JobHistory Server is bad. The following health tests are bad: host health.
The health test result for JOBHISTORY_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Role: JobHistory Server (hadoop103)
Role Type: JobHistory Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
JOBHISTORY_HOST_HEALTH Role health test bad Critical The health test result for JOBHISTORY_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Role: Server (hadoop103)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for IMPALA_STATESTORE_HEALTH has become bad: The health of the Impala StateStore is bad. The following health tests are bad: host health.
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Health Test Results:
Health Test Name Event Code Severity Content
IMPALA_STATESTORE_HEALTH Service health test bad Critical The health test result for IMPALA_STATESTORE_HEALTH has become bad: The health of the Impala StateStore is bad. The following health tests are bad: host health.
The health test result for STATESTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Role: Impala StateStore (hadoop103)
Role Type: Impala StateStore
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
STATESTORE_HOST_HEALTH Role health test bad Critical The health test result for STATESTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Role: JournalNode (hadoop103)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_HOST_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Role: kafka_broker (hadoop103)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HOST_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HUE_LOAD_BALANCER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Role: Load Balancer (hadoop103)
Role Type: Load Balancer
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
HUE_LOAD_BALANCER_HOST_HEALTH Role health test bad Critical The health test result for HUE_LOAD_BALANCER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The health of the SPARK_YARN_HISTORY_SERVER is bad. The following health tests are bad: host health.
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: spark_on_yarn
Service Display Name: Spark
Service Type: SPARK_ON_YARN
Health Test Results:
Health Test Name Event Code Severity Content
SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH Service health test bad Critical The health test result for SPARK_ON_YARN_SPARK_ON_YARN_SPARK_YARN_HISTORY_SERVER_HEALTH has become bad: The health of the SPARK_YARN_HISTORY_SERVER is bad. The following health tests are bad: host health.
Health test changes: 2 Became Bad
Time: Aug 8, 2024 10:02:10 AM
Monitor Startup: false
Role: Impala Daemon (hadoop103)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop103
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_QUERY_MONITORING_STATUS Role health test bad Critical The health test result for IMPALAD_QUERY_MONITORING_STATUS has become bad: There are 6 error(s) seen monitoring executing queries, and 0 errors(s) seen monitoring completed queries for this role in the previous 5 minute(s). Critical threshold: any.
IMPALAD_HOST_HEALTH Role health test bad Critical The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HIVEMETASTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: Hive Metastore Server (hadoop101)
Role Type: Hive Metastore Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HIVEMETASTORE_HOST_HEALTH Role health test bad Critical The health test result for HIVEMETASTORE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HUE_LOAD_BALANCER_HEALTHY has become bad: Healthy Load Balancer: 0. Concerning Load Balancer: 0. Total Load Balancer: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Health Test Results:
Health Test Name Event Code Severity Content
HUE_LOAD_BALANCER_HEALTHY Service health test bad Critical The health test result for HUE_LOAD_BALANCER_HEALTHY has become bad: Healthy Load Balancer: 0. Concerning Load Balancer: 0. Total Load Balancer: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: NodeManager (hadoop101)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Health test changes: 2 Became Bad
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_DATA_NODES_HEALTHY Service health test bad Critical The health test result for HDFS_DATA_NODES_HEALTHY has become bad: Healthy DataNode: 2. Concerning DataNode: 0. Total DataNode: 3. Percent healthy: 66.67%. Percent healthy or concerning: 66.67%. Critical threshold: 90.00%.
HDFS_FAILOVER_CONTROLLERS_HEALTHY Service health test bad Critical The health test result for HDFS_FAILOVER_CONTROLLERS_HEALTHY has become bad: Failover Controllers with bad health: Hadoop101. The following health tests are bad: host health.
The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: Failover Controller (hadoop101)
Role Type: Failover Controller
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVERCONTROLLER_HOST_HEALTH Role health test bad Critical The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HIVESERVER2_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: HiveServer2 (hadoop101)
Role Type: HiveServer2
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HIVESERVER2_HOST_HEALTH Role health test bad Critical The health test result for HIVESERVER2_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: NameNode (hadoop101)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_HOST_HEALTH Role health test bad Critical The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for YARN_NODE_MANAGERS_HEALTHY has become bad: Healthy NodeManager: 2. Concerning NodeManager: 0. Total NodeManager: 3. Percent healthy: 66.67%. Percent healthy or concerning: 66.67%. Critical threshold: 90.00%.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Health Test Results:
Health Test Name Event Code Severity Content
YARN_NODE_MANAGERS_HEALTHY Service health test bad Critical The health test result for YARN_NODE_MANAGERS_HEALTHY has become bad: Healthy NodeManager: 2. Concerning NodeManager: 0. Total NodeManager: 3. Percent healthy: 66.67%. Percent healthy or concerning: 66.67%. Critical threshold: 90.00%.
The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: Server (hadoop101)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HOST_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: Host Monitor (hadoop101)
Role Type: Host Monitor
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
HOST_MONITOR_HOST_HEALTH Role health test bad Critical The health test result for HOST_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for IMPALA_IMPALADS_HEALTHY has become bad: Healthy Impala Daemon: 2. Concerning Impala Daemon: 0. Total Impala Daemon: 3. Percent healthy: 66.67%. Percent healthy or concerning: 66.67%. Critical threshold: 90.00%.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Health Test Results:
Health Test Name Event Code Severity Content
IMPALA_IMPALADS_HEALTHY Service health test bad Critical The health test result for IMPALA_IMPALADS_HEALTHY has become bad: Healthy Impala Daemon: 2. Concerning Impala Daemon: 0. Total Impala Daemon: 3. Percent healthy: 66.67%. Percent healthy or concerning: 66.67%. Critical threshold: 90.00%.
The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: JournalNode (hadoop101)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_HOST_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: kafka_broker (hadoop101)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HOST_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for ALERT_PUBLISHER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: Alert Publisher (hadoop101)
Role Type: Alert Publisher
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
ALERT_PUBLISHER_HOST_HEALTH Role health test bad Critical The health test result for ALERT_PUBLISHER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for SERVICE_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: Service Monitor (hadoop101)
Role Type: Service Monitor
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
SERVICE_MONITOR_HOST_HEALTH Role health test bad Critical The health test result for SERVICE_MONITOR_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Health test changes: 2 Became Bad
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: Impala Daemon (hadoop101)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_QUERY_MONITORING_STATUS Role health test bad Critical The health test result for IMPALAD_QUERY_MONITORING_STATUS has become bad: There are 6 error(s) seen monitoring executing queries, and 0 errors(s) seen monitoring completed queries for this role in the previous 5 minute(s). Critical threshold: any.
IMPALAD_HOST_HEALTH Role health test bad Critical The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for EVENT_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:15 AM
Monitor Startup: false
Role: Event Server (hadoop101)
Role Type: Event Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: mgmt
Service Display Name: Cloudera Management Service
Service Type: Cloudera Management Service
Hosts: Hadoop101
Health Test Results:
Health Test Name Event Code Severity Content
EVENT_SERVER_HOST_HEALTH Role health test bad Critical The health test result for EVENT_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for ZOOKEEPER_SERVERS_HEALTHY has become bad: Healthy Server: 1. Concerning Server: 0. Total Server: 3. Percent healthy: 33.33%. Percent healthy or concerning: 33.33%. Critical threshold: 51.00%.
Time: Aug 8, 2024 10:02:20 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVERS_HEALTHY Service health test bad Critical The health test result for ZOOKEEPER_SERVERS_HEALTHY has become bad: Healthy Server: 1. Concerning Server: 0. Total Server: 3. Percent healthy: 33.33%. Percent healthy or concerning: 33.33%. Critical threshold: 51.00%.
The health test result for KAFKA_KAFKA_BROKER_HEALTHY has become bad: Healthy KAFKA_BROKER: 1. Concerning KAFKA_BROKER: 0. Total KAFKA_BROKER: 3. Percent healthy: 33.33%. Percent healthy or concerning: 33.33%. Critical threshold: 49.99%.
Time: Aug 8, 2024 10:02:20 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HEALTHY Service health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HEALTHY has become bad: Healthy KAFKA_BROKER: 1. Concerning KAFKA_BROKER: 0. Total KAFKA_BROKER: 3. Percent healthy: 33.33%. Percent healthy or concerning: 33.33%. Critical threshold: 49.99%.
From noreply@localhost.localdomain Thu Aug 8 10:03:15 2024 Return-Path: X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Received: from Hadoop101 (localhost [127.0.0.1]) by Hadoop101.localdomain (Postfix) with ESMTP id 54DDE42A4169 for ; Thu, 8 Aug 2024 10:03:15 +0800 (CST) Date: Thu, 8 Aug 2024 10:03:15 +0800 (CST) From: noreply@localhost.localdomain To: root@localhost.localdomain Message-ID: <1797363767.23.1723082595346@localhost> Subject: [Cloudera Alert] 14 Alerts since 10:02:20 AM MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit breadcrumbId: ID-Hadoop101-40818-1721292560607-0-24081 [Cloudera Alert] 14 Alerts since 10:02:20 AM
Health test changes: 2 Became Bad
Time: Aug 8, 2024 10:02:20 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hive
Service Display Name: Hive
Service Type: Hive
Health Test Results:
Health Test Name Event Code Severity Content
HIVE_HIVESERVER2S_HEALTHY Service health test bad Critical The health test result for HIVE_HIVESERVER2S_HEALTHY has become bad: Healthy HiveServer2: 0. Concerning HiveServer2: 0. Total HiveServer2: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
HIVE_HIVEMETASTORES_HEALTHY Service health test bad Critical The health test result for HIVE_HIVEMETASTORES_HEALTHY has become bad: Healthy Hive Metastore Server: 0. Concerning Hive Metastore Server: 0. Total Hive Metastore Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:30 AM
Monitor Startup: false
Role: Server (hadoop102)
Role Type: Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: zookeeper
Service Display Name: ZooKeeper
Service Type: ZooKeeper
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
ZOOKEEPER_SERVER_HOST_HEALTH Role health test bad Critical The health test result for ZOOKEEPER_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HUE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:30 AM
Monitor Startup: false
Role: Hue Server (hadoop102)
Role Type: Hue Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
HUE_SERVER_HOST_HEALTH Role health test bad Critical The health test result for HUE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:30 AM
Monitor Startup: false
Role: kafka_broker (hadoop102)
Role Type: KAFKA_BROKER
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: kafka
Service Display Name: Kafka
Service Type: KAFKA
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
KAFKA_KAFKA_BROKER_HOST_HEALTH Role health test bad Critical The health test result for KAFKA_KAFKA_BROKER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for OOZIE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:30 AM
Monitor Startup: false
Role: Oozie Server (hadoop102)
Role Type: Oozie Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: oozie
Service Display Name: Oozie
Service Type: Oozie
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
OOZIE_SERVER_HOST_HEALTH Role health test bad Critical The health test result for OOZIE_SERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:30 AM
Monitor Startup: false
Role: JournalNode (hadoop102)
Role Type: JournalNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
JOURNAL_NODE_HOST_HEALTH Role health test bad Critical The health test result for JOURNAL_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:30 AM
Monitor Startup: false
Role: Impala Daemon (hadoop102)
Role Type: Impala Daemon
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
IMPALAD_HOST_HEALTH Role health test bad Critical The health test result for IMPALAD_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for IMPALA_CATALOGSERVER_HEALTH has become bad: The health of the Impala Catalog Server is bad. The following health tests are bad: host health.
Time: Aug 8, 2024 10:02:30 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Health Test Results:
Health Test Name Event Code Severity Content
IMPALA_CATALOGSERVER_HEALTH Service health test bad Critical The health test result for IMPALA_CATALOGSERVER_HEALTH has become bad: The health of the Impala Catalog Server is bad. The following health tests are bad: host health.
The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:30 AM
Monitor Startup: false
Role: NodeManager (hadoop102)
Role Type: NodeManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NODE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for NODE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:30 AM
Monitor Startup: false
Role: ResourceManager (hadoop102)
Role Type: ResourceManager
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: yarn
Service Display Name: YARN (MR2 Included)
Service Type: YARN (MR2 Included)
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
RESOURCE_MANAGER_HOST_HEALTH Role health test bad Critical The health test result for RESOURCE_MANAGER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:30 AM
Monitor Startup: false
Role: Failover Controller (hadoop102)
Role Type: Failover Controller
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
HDFS_FAILOVERCONTROLLER_HOST_HEALTH Role health test bad Critical The health test result for HDFS_FAILOVERCONTROLLER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:30 AM
Monitor Startup: false
Role: NameNode (hadoop102)
Role Type: NameNode
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hdfs
Service Display Name: HDFS
Service Type: HDFS
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
NAME_NODE_HOST_HEALTH Role health test bad Critical The health test result for NAME_NODE_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for CATALOGSERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
Time: Aug 8, 2024 10:02:30 AM
Monitor Startup: false
Role: Impala Catalog Server (hadoop102)
Role Type: Impala Catalog Server
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: impala
Service Display Name: Impala
Service Type: Impala
Hosts: Hadoop102
Health Test Results:
Health Test Name Event Code Severity Content
CATALOGSERVER_HOST_HEALTH Role health test bad Critical The health test result for CATALOGSERVER_HOST_HEALTH has become bad: The health of this role's host is bad. The following health tests are bad: clock offset.
The health test result for HUE_HUE_SERVERS_HEALTHY has become bad: Healthy Hue Server: 0. Concerning Hue Server: 0. Total Hue Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.
Time: Aug 8, 2024 10:02:35 AM
Monitor Startup: false
Cluster: Cluster 1
Cluster Display Name: Cluster 1
Service: hue
Service Display Name: Hue
Service Type: Hue
Health Test Results:
Health Test Name Event Code Severity Content
HUE_HUE_SERVERS_HEALTHY Service health test bad Critical The health test result for HUE_HUE_SERVERS_HEALTHY has become bad: Healthy Hue Server: 0. Concerning Hue Server: 0. Total Hue Server: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.