Wednesday 25 June 2014

Shell and Email Actions in Oozie

1. Shell action:
ERROR: Cannot run program "script.sh" (in directory "/mapred/local/taskTracker/dell/jobcache/job_201312061003_0001/attempt_201312061003_0001_m_000000_0/work"): java.io.IOException: error=2, No such file or directory.

An Oozie shell action is executed on a random Hadoop node, i.e. not locally on the machine where the Oozie server is running. Make sure that your script is on the node that executes the job.

<action name="shellAction">
<shell xmlns="uri:oozie:shell-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
            <delete path='${rawDataPath}'/>
        </prepare>
<configuration>
   <property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
   </property>
</configuration>
<exec>${sqoopScript}</exec>
<argument>${wf:user()}</argument>
<file>${sqoopScriptPath}#${sqoopScript}</file>
<capture-output/>
</shell>
<ok to="pigAction"/>
<error to="killAction"/>
    </action>


(1) sqoopScriptPath = path/filename
      sqoopScript= filename
      The other files, e.g .jar need to be placed in <file>tags.
(2) Shell's parameters can be passed in as a similar way as in command line.
 <argument>${wf:user()}</argument> should be placed before <file>
In shell, use $1, $2 to refer the parameters.


2. Email Action Configuration

The email action requires some SMTP server configuration to be present (in oozie-site.xml). The following are the values it looks for:

oozie.email.smtp.host - The host where the email action may find the SMTP server (localhost by default).
oozie.email.smtp.port - The port to connect to for the SMTP server (25 by default). oozie.email.from.address - The from address to be used for mailing all emails (oozie@localhost by default).
oozie.email.smtp.auth - Boolean property that toggles if authentication is to be done or not. (false by default).
oozie.email.smtp.username - If authentication is enabled, the username to login as (empty by default).
oozie.email.smtp.password - If authentication is enabled, the username's password (empty by default).

<action name="[NODE-NAME]">
   <email xmlns="uri:oozie:email-action:0.1">
     <to>[COMMA-SEPARATED-TO-ADDRESSES]</to>
     <cc>[COMMA-SEPARATED-CC-ADDRESSES]</cc> <!-- cc is optional -->
     <subject>[SUBJECT]</subject>
     <body>[BODY]</body>
   </email>
   <ok to="[NODE-NAME]"/>
   <error to="[NODE-NAME]"/>
</action>


Reference:
http://oozie.apache.org/docs/3.3.0/DG_EmailActionExtension.html
https://github.com/airawat/OozieSamples/blob/master/oozieProject/workflowShellAction/workflow.xml

No comments:

Post a Comment