Details
-
Type:
Bug
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 1.1.0
-
Fix Version/s: None
-
Component/s: import
-
Labels:None
-
Environment:Ubuntu 10.04 on Amazon EC2, with CDH3b3
Description
When importing large table (60M rows) with command:
sqoop import -D mapred.task.timeout=0 --connect "jdbc:mysql://
my.db.com/$dbname?netTimeoutForStreamingResults=3600" -z --username
$username P --table $tablename --hbase-table $tablename --column
family default --hbase-create-table
it would fail with following messages, but importing the same table to hive/hdfs never have any problem.
==========================================================================================
11/02/11 10:02:29 INFO mapred.JobClient: Task Id :
attempt_201012180743_0028_m_000001_0, Status : FAILED
java.io.IOException: SQLException in nextKeyValue
at
com.cloudera.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:
238)
at org.apache.hadoop.mapred.MapTask
$NewTrackingRecordReader.nextKeyValue(MapTask.java:448)
at
org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:
67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at
com.cloudera.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:
187)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:
639)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:315)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:
1063)
at org.apache.hadoop.mapred.Child.main(Child.java:211)
Caused by: com.mysql.jdbc.CommunicationsException: Communications link
failure due to underlying exception:
-
- BEGIN NESTED EXCEPTION **
java.io.EOFException
MESSAGE: Can not read response from server. Expected to read 822
bytes, read 572 bytes before connection was unexpectedly lost.
STACKTRACE:
java.io.EOFException: Can not read response from server. Expected to
read 822 bytes, read 572 bytes before connection was unexpectedly
lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1997)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:
2464)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2916)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:885)
at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1360)
at
com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:366)
at com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:355)
at com.mysql.jdbc.ResultSet.next(ResultSet.java:7302)
at
com.cloudera.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:
227)
at org.apache.hadoop.mapred.MapTask
$NewTrackingRecordReader.nextKeyValue(MapTask.java:448)
at
org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:
67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:
143) at
com.cloudera.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:
187)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:
639)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:315)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:
1063)
at org.apache.hadoop.mapred.Child.main(Child.java:211) - END NESTED EXCEPTION **
Last packet sent to the server was 349784 ms ago.
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:
2622)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2916)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:885)
at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1360)
at
com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:366)
at com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:355)
at com.mysql.jdbc.ResultSet.next(ResultSet.java:7302)
at
com.cloudera.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:
227)
... 11 more
- BEGIN NESTED EXCEPTION **