1. Storing the customized jar in "lib" directory
Jars in the lib directory of Pig script are automatically added to the classpath by Oozie server. So, if the customized jar is in lib directory, then the jar file should not be in
However, we use Hue to design workflow. The lib path must be specify in an absolute path.
For example, hdfs://localhost:8020/user/training/xxx.jar
If using Oozie commands, "REGISTER" can be removed from Pig script.
Reference:
https://cwiki.apache.org/confluence/display/OOZIE/Pig+Cookbook
<archive>
and hence, "REGISTER" should be removed from Pig script.However, we use Hue to design workflow. The lib path must be specify in an absolute path.
For example, hdfs://localhost:8020/user/training/xxx.jar
If using Oozie commands, "REGISTER" can be removed from Pig script.
Reference:
https://cwiki.apache.org/confluence/display/OOZIE/Pig+Cookbook
2. Storing the parameter file in "lib" directory
pig -file script.pig -param INPUT=inputdir -param OUTPUT=outputdir
The parameter file can be stored in the lib directory as contents of this directory are automatically added to the classpath by Oozie server.
Partial xml file for Pig action:
No comments:
Post a Comment