Details
-
Type:
Bug
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: CDH 5.3.0
-
Fix Version/s: None
-
Component/s: Sqoop
-
Labels:None
-
Environment:Client: Windows 7, Pentaho 5.3.0.1
Cluster: CDH5.3 Secured cluster
Description
running sqoop 1.4.5 import from MySql to Hbase from Pentaho.
the tool is run not from command line but using Sqoop.runTool() API.
The parameters are the following:
--split-by client_ip --column-family client_ip --hbase-create-table --hbase-table hbasesqooptest2 --connect jdbc:mysql://10.177.176.105:3306/aggregatehdfs?defaultFetchSize=500&useCursorFetch=true --username USER --password PASSWORD --table aggregatehdfs
after submitting MR job sqoop hangs for long time then fails.
after the sqoop-1.4.5-cdh5.3.0.jar had been replaced with sqoop-1.4.5-hadoop23.jar (original apache version) the job was completed successfully.
the archive with log is attached. Please pay attention to the following lines:
For Cloudera's version
Submitting tokens for job: job_1428070790584_0001 Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:cdh53secure, Ident: (HDFS_DELEGATION_TOKEN token 318 for devuser)
For Apache's version:
Submitting tokens for job: job_1428070790584_0002 Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:cdh53secure, Ident: (HDFS_DELEGATION_TOKEN token 319 for devuser) Kind: HBASE_AUTH_TOKEN, Service: c8d5f8fa-a408-440d-8385-59aa245faa1e, Ident: (org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier@9)
it can be seen that the second version submits HBASE AUTH TOKEN while the first one doesn't
briefly flicking through sources it was noticed there is no check:
if (!isAuthenticated(admin))
in the org.apache.sqoop.mapreduce.HBaseImportJob.jobSetup() method for the Apache's version of the class.