Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Not A Bug
-
Affects Version/s: 3.10.0
-
Fix Version/s: None
-
Component/s: con.pig
-
Labels:None
-
Environment:
Hue 3.10.0, Hive 2.1.0, tez 0.8.3, Pig 0.15.0
Description
With Hue 3.10.0, the following tools seems to not take Tez as default execution engine (Pig only) even when this option is set:
- Pig Editor
- Notebooks
- Oozie Pig workflow
An example of Pig script:
SET exectype tez;
SET tez.queue.name 'HQ_OLPS';
CSV_HDFS = LOAD 'hdfs://sandbox/user/pierre/WORK/chicago_crimes_2001_to_present.csv' USING org.apache.pig.piggybank.storage.CSVExcelStorage(',', 'NO_MULTILINE', 'UNIX', 'SKIP_INPUT_HEADER') AS (id:long, casenumber:chararray, date:chararray, block:chararray, iucr:chararray, primarytype:chararray, description:chararray, locationdescription:chararray, arrest:boolean, domestic:boolean, beat:int, district:int, ward:int, communityarea:int, fbicode:chararray, xcoordinate:int, ycoordinate:int, year:int, updatedon:chararray, latitude:chararray, longitude:chararray);
LIMIT_CSV = LIMIT CSV_HDFS 1000;
CSV_GRP = GROUP LIMIT_CSV BY year;
DUMP CSV_GRP;
The job runs on MapReduce instead of Tez. I tried to add the property in "Hadoop properties" but this doesn't change the behavior. Note: setting Tez as the default execution engine works well with Hive.
