Tuesday, 24 June 2014

Pig Action in Oozie

1. Storing the customized jar in "lib" directory
Jars in the lib directory of Pig script are automatically added to the classpath by Oozie server. So, if the customized jar is in lib directory, then the jar file should not be in <archive> and hence, "REGISTER" should be removed from Pig script.

However, we use Hue to design workflow. The lib path must be specify in an absolute path.
For example, hdfs://localhost:8020/user/training/xxx.jar

If using Oozie commands, "REGISTER" can be removed from Pig script.


Reference:
https://cwiki.apache.org/confluence/display/OOZIE/Pig+Cookbook
2. Storing the parameter file in "lib" directory
pig -file script.pig -param INPUT=inputdir -param OUTPUT=outputdir
The parameter file can be stored in the lib directory as contents of this directory are automatically added to the classpath by Oozie server.
Partial xml file for Pig action:
<script>script.pig</script>
<argument>-param_file</argument>
<argument>paramfile.txt</argument>

No comments:

Post a Comment