Details
-
Type:
Improvement
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: CDH 5.12.0
-
Component/s: Hadoop Common
-
Labels:None
Description
CDH versions all through currently-latest cdh5.9.1 ship with Amazon's aws-sdk-java 1.10.6. This version is from July 2015 and there have been many releases since then.
As of now (January 2017) the latest aws-sdk-ajava version is 1.11.86.
Certainly there have been many fixes since then. Particularly, we are seeing issues due to multipart-related bug that was fixed back in 1.10.38 (https://aws.amazon.com/releasenotes/4924952904399814 - "Merged Github pull request #545. Default Content-Type to application/octet-stream.") that is causing issues when using s3a:// URLs with IBM's Cloud Object Storage (https://ibm-public-cos.github.io/crs-docs/).