Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 1.1.0
    • Fix Version/s: None
    • Component/s: hive, import
    • Labels:
      None

      Description

      When attempting to import a MySQL database with the option --hive-overwrite the job fails. It attempts to create the hive table directory, but it already exists. If the --hive-overwrite option is given it should simply overwrite the directory.

      10/10/29 15:26:20 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
      10/10/29 15:26:20 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
      10/10/29 15:26:20 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
      10/10/29 15:26:20 INFO tool.CodeGenTool: Beginning code generation
      10/10/29 15:26:20 INFO manager.MySQLManager: Executing SQL statement: SELECT t.* FROM `addresses_deals` AS t LIMIT 1
      10/10/29 15:26:20 INFO orm.CompilationManager: HADOOP_HOME is /home/dobby/root/hadoop-0.20.2+737
      Note: /tmp/sqoop-dobby/compile/ea3c2eef1df30e62fd9a0cd83eeb7cbb/addresses_deals.java uses or overrides a deprecated API.
      Note: Recompile with -Xlint:deprecation for details.
      10/10/29 15:26:21 ERROR orm.CompilationManager: Could not rename /tmp/sqoop-dobby/compile/ea3c2eef1df30e62fd9a0cd83eeb7cbb/addresses_deals.java to /home/dobby/control/sqoop/./tmp/java/deals_production_addresses_deals/addresses_deals.java
      10/10/29 15:26:21 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-dobby/compile/ea3c2eef1df30e62fd9a0cd83eeb7cbb/addresses_deals.jar
      10/10/29 15:26:21 WARN manager.DirectMySQLManager: Direct-mode import from MySQL does not support column
      10/10/29 15:26:21 WARN manager.DirectMySQLManager: selection. Falling back to JDBC-based import.
      10/10/29 15:26:21 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
      10/10/29 15:26:21 INFO mapreduce.ImportJobBase: Beginning import of addresses_deals
      10/10/29 15:26:22 INFO mapred.JobClient: Cleaning up the staging area hdfs://b63i:9000/tmp/hadoop-dobby/mapred/staging/dobby/.staging/job_201010291502_0006
      10/10/29 15:26:22 ERROR tool.ImportTool: Encountered IOException running import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /user/hive/warehouse/addresses_deals already exists
      at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:132)
      at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:827)
      at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:793)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
      at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:793)
      at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
      at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:495)
      at com.cloudera.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:107)
      at com.cloudera.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:166)
      at com.cloudera.sqoop.manager.SqlManager.importTable(SqlManager.java:385)
      at com.cloudera.sqoop.manager.MySQLManager.importTable(MySQLManager.java:125)
      at com.cloudera.sqoop.manager.DirectMySQLManager.importTable(DirectMySQLManager.java:59)
      at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:350)
      at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
      at com.cloudera.sqoop.Sqoop.run(Sqoop.java:134)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
      at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:170)
      at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:196)
      at com.cloudera.sqoop.Sqoop.main(Sqoop.java:205)

        Attachments

        1. SQOOP-122.patch
          2 kB
          Brian Muller
        2. SQOOP-122.2.patch
          2 kB
          Brian Muller
        3. SQOOP-122.3.patch
          3 kB
          Lars Francke

          Activity

            People

            • Assignee:
              bmuller Brian Muller
              Reporter:
              bmuller Brian Muller
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: