Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 0.17.0
-
Fix Version/s: 0.17.0
-
Component/s: None
-
Labels:None
Description
When I tried to use deflate as compression type for Parquet format, I noticed that it will always create a snappy encoded dataset. The root cause is that I do not specify the new property location, so that Kite tries to suggest one for me. And it will create a new DatasetDescriptor. And it seems two properties of descriptor are not copied.
private ColumnMapping columnMapping; private CompressionType compressionType; public Builder(DatasetDescriptor descriptor) { this.schema = descriptor.getSchema(); this.schemaUrl = descriptor.getSchemaUrl(); this.format = descriptor.getFormat(); this.properties = Maps.newHashMap(descriptor.properties); this.location = descriptor.getLocation(); if (descriptor.isPartitioned()) { this.partitionStrategy = descriptor.getPartitionStrategy(); } }