Uploaded image for project: 'Kite SDK'
  1. Kite SDK
  2. KITE-657

Schema URI call to makeQualified incorrectly copies authority

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.17.0
    • Fix Version/s: 0.17.0
    • Component/s: None
    • Labels:
      None

      Description

      I think this is a but in Path, but we need a work-around. When makeQualified is called on a file URI and the default FS is a HDFS URI, the authority section from the HDFS URI is incorrectly copied to the file path. This causes dataset creation to fail:

      [cloudera@quickstart ~]$ ./dataset -v create events -s event.avsc                                                                                                                                                    
      Argument error
      java.lang.IllegalArgumentException: Wrong FS: file://quickstart.cloudera:8020/home/cloudera/event.avsc, expected: file:///
              at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:643)
              at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:79)
              at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:506)
              at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:724)
              at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:501)
              at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
              at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:137)
              at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:339)
              at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764)
              at org.kitesdk.data.DatasetDescriptor$Builder.open(DatasetDescriptor.java:962)
              at org.kitesdk.data.DatasetDescriptor$Builder.schemaUri(DatasetDescriptor.java:452)
              at org.kitesdk.cli.commands.CreateDatasetCommand.run(CreateDatasetCommand.java:81)
              at org.kitesdk.cli.Main.run(Main.java:154)
              at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
              at org.kitesdk.cli.Main.main(Main.java:232)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
      

      This was introduced by CDK-639.

        Attachments

          Activity

            People

            • Assignee:
              blue Ryan Blue
              Reporter:
              blue Ryan Blue
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: