Uploaded image for project: 'Kite SDK (READ-ONLY)'
  1. Kite SDK (READ-ONLY)
  2. KITE-791

Schema validations should allow fields prefixed with an underscore

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.17.0
    • Fix Version/s: 0.17.1
    • Component/s: Data Module
    • Labels:
      None

      Description

      Originally ran into this with SQOOP-1780 when I was trying to import some database tables whose columns start with a number into Parquet, but we correctly identified this as a Kite issue:

      java.lang.IllegalStateException: Deprecated: field names are not alphanumeric (plus '_'): sqoop_import_team._1QP, sqoop_import_team._2QP, sqoop_import_team._3QP, sqoop_import_team._4QP
      at com.google.common.base.Preconditions.checkState(Preconditions.java:172)
      at org.kitesdk.data.spi.Compatibility.checkSchema(Compatibility.java:119)
      at org.kitesdk.data.spi.Compatibility.checkDescriptor(Compatibility.java:133)
      at org.kitesdk.data.spi.hive.HiveManagedMetadataProvider.create(HiveManagedMetadataProvider.java:40)
      at org.kitesdk.data.spi.hive.HiveManagedDatasetRepository.create(HiveManagedDatasetRepository.java:76)
      at org.kitesdk.data.Datasets.create(Datasets.java:200)
      at org.kitesdk.data.Datasets.create(Datasets.java:240)
      at org.apache.sqoop.mapreduce.ParquetJob.createDataset(ParquetJob.java:81)
      at org.apache.sqoop.mapreduce.ParquetJob.configureImportJob(ParquetJob.java:70)

        Attachments

          Activity

            People

            • Assignee:
              tom Tom White
              Reporter:
              jwills Josh Wills
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: