Friday 8 August 2014

Timing your Pig UDFs



SET pig.udf.profile true;

 will turn on counters that approximately measure the number of invocations and microseconds spent in all UDFs and Loaders. Use this with caution, as this feature can really bloat the number of counters your job uses! Useful for lightweight debugging of jobs, not for big dataset in production.



Reference:
https://pig.apache.org/docs/r0.11.1/perf.html#profiling


No comments:

Post a Comment