Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 0.17.0
-
Fix Version/s: 0.17.0
-
Component/s: None
-
Labels:None
Description
I think this is a but in Path, but we need a work-around. When makeQualified is called on a file URI and the default FS is a HDFS URI, the authority section from the HDFS URI is incorrectly copied to the file path. This causes dataset creation to fail:
[cloudera@quickstart ~]$ ./dataset -v create events -s event.avsc
Argument error
java.lang.IllegalArgumentException: Wrong FS: file://quickstart.cloudera:8020/home/cloudera/event.avsc, expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:643)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:79)
at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:506)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:724)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:501)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:137)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:339)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764)
at org.kitesdk.data.DatasetDescriptor$Builder.open(DatasetDescriptor.java:962)
at org.kitesdk.data.DatasetDescriptor$Builder.schemaUri(DatasetDescriptor.java:452)
at org.kitesdk.cli.commands.CreateDatasetCommand.run(CreateDatasetCommand.java:81)
at org.kitesdk.cli.Main.run(Main.java:154)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.kitesdk.cli.Main.main(Main.java:232)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
This was introduced by CDK-639.