Details
-
Type: Task
-
Status: Resolved
-
Priority: Major
-
Resolution: Fixed
-
Affects Version/s: 0.15.0
-
Fix Version/s: 0.16.0
-
Component/s: None
-
Labels:None
Description
A partner trying to write to parquet on the CDH 5.0.0-1 VM was having jobs fail with OutOfMemoryExceptions. Even turning down the writer cache to just one writer still hit the problem because the parquet block size was too large for one writer. We should pass any parquet.* options to the ParquetInputFormat when configuring MR.