Monday 28 July 2014

NameNode Change after upgrade HA for CDH4

After upgrading HA for CDH4.7, we encountered an error for oozie job.

It is because the name node should be set as 'nameservice1'

FAILED: SemanticException [Error 10028]: Line 2:17 Path is not legal
''hdfs://upladevhdn01.researchnow.com:8020/user/silverdollar/RawPassiveData
Uploads'': Move from:
        hdfs://upladevhdn01.researchnow.com:8020/user/silverdollar/RawPassiveDataU
ploads to:
hdfs://nameservice1/user/hive/warehouse/passivedatauploads_oozie_test is
not valid. Please check that    values for params "default.fs.name" and
"hive.metastore.warehouse.dir" do not conflict.

Configuring Oozie to Use HDFS HA

To configure an Oozie workflow to use HDFS HA, use the HA HDFS URI instead of the NameNode URI in the <name-node> element of the workflow.
Example:
<action name="mr-node">
  <map-reduce>
    <job-tracker>${jobTracker}</job-tracker>
    <name-node>hdfs://ha-nn
where ha-nn is the value of dfs.nameservices in hdfs-site.xml.

Upgrading the Hive Metastore to Use HDFS HA

  • hdfs://nameservice1 specifies the new location and should match the value of the dfs.nameservicesproperty.

Reference:
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-High-Availability-Guide/cdh4hag_topic_2_6.html

No comments:

Post a Comment