Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Wiki Markup
h1. Additional Configuration for YARN Shims

# {color:#333e48}Set the{color} [active Hadoop distribution|../../../../../../../../../display/BAD/Configuring+Pentaho+for+your+Hadoop+Distro+and+Version]{color:#333e48}.{color}
# Configure the cluster settings.
# {color:#333e48}Navigate to the folder that contains the shim, then open the yarn-site.xml file in a text editor.{color}
# {color:#333e48}Check the value for the yarn.application.classpath. The yarn.application.classpath specifies the classpath to use on the cluster for executing YARN applications. If the paths for your environment are not listed, add them. Paths are separated by a comma.{color}
# {color:#333e48}Check the value for the yarn.resourcemanager.hostname.{color}
\* {color:#333e48}{*}For CDH 5{*}{color}{color:#333e48}, the default is clouderamanager.cdh5.test.{color} 
\* {color:#333e48}{*}For HDP 2{*}{color}{color:#333e48}, the{color}{color:#333e48} {color}{color:#333e48}default value is sandbox.hortonworks.com.{color}
\*&nbsp;*For all YARN shims:&nbsp;*{color:#333e48}Since your hostname will probably be different, make the change to the value.{color}{color:#333e48}&nbsp;&nbsp;{color}<value>yarncluster.mycompany.com</value>
# {color:#333e48}{*}For HDP 2{*}{color}{color:#333e48}, check the value for the yarn.resourcemanager.address and change the port to match your environment if necessary like this: &nbsp;{color}<value>${yarn.resourcemanager.hostname}:9000</value>
# {color:#333e48}If you are using CDH 5.x, there are a few more things that you need to do:{color}
#* {color:#333e48}Navigate to the folder that contains the shim, then open the hive-site.xml file in a text editor.{color}&nbsp;{color:#333e48}Modify the hive.metastore.uris property so that it points to the location of your hive metastore.{color}&nbsp;{color:#333e48}Save and close the hive-site.xml file.{color}
#* {color:#333e48}Navigate to the folder that contains the shim, then open the mapred-site.xml file, in a text editor.{color}&nbsp;{color:#333e48}Modify the mapreduce.jobhistory.address property so that it points to the place where the job history logs are stored.{color}

{color:#333e48}Note: Not all shim properties can be set in the Spoon user interface, nor are instructions for modifying them listed here. If you need to set additional properties that are not addressed, you will need to set them manually in the \*-site.xml files that are in the shim directory. Consult your Hadoop distribution's vendor for details about the properties you want to set.{color}