Details
-
Type:
Bug
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: CDH 5.4.2
-
Fix Version/s: None
-
Component/s: Oozie
-
Labels:None
-
Environment:CentOS 6.6
Description
After working HDFS HA setup:
- hdfs dfs -ls hdfs://warehouse/
Found 3 items
drwxrwxrwt - hdfs supergroup 0 2015-06-16 12:58 hdfs://warehouse/tmp
drwxr-xr-x - hdfs supergroup 0 2015-06-16 13:09 hdfs://warehouse/user
drwxr-xr-x - hdfs supergroup 0 2015-06-16 12:53 hdfs://warehouse/var
Ozie fails to install shared libs:
- oozie-setup sharelib create -fs hdfs://warehouse:8020 -locallib /usr/lib/oozie/oozie-sharelib-yarn
setting JAVA_LIBRARY_PATH="$JAVA_LIBRARY_PATH:/usr/lib/hadoop/lib/native"
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
setting OOZIE_HTTPS_PORT=11443
setting OOZIE_HTTPS_KEYSTORE_PASS=password
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.port=${OOZIE_HTTPS_PORT}"
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.keystore.pass=${OOZIE_HTTPS_KEYSTORE_PASS}"
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
setting JAVA_LIBRARY_PATH="$JAVA_LIBRARY_PATH:/usr/lib/hadoop/lib/native"
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
setting OOZIE_HTTPS_PORT=11443
setting OOZIE_HTTPS_KEYSTORE_PASS=password
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.port=${OOZIE_HTTPS_PORT}"
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.keystore.pass=${OOZIE_HTTPS_KEYSTORE_PASS}"
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/oozie/libtools/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/oozie/libserver/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
the destination path for sharelib is: /user/oozie/share/lib/lib_20150616132914
Error: java.net.UnknownHostException: warehouse
Stack trace for the error was (for debug purposes):
--------------------------------------
java.lang.IllegalArgumentException: java.net.UnknownHostException: warehouse
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:374)
at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:312)
at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:178)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:664)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:608)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:148)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
at org.apache.oozie.service.HadoopAccessorService$2.run(HadoopAccessorService.java:423)
at org.apache.oozie.service.HadoopAccessorService$2.run(HadoopAccessorService.java:421)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:421)
at org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:159)
at org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:57)
Caused by: java.net.UnknownHostException: warehouse
... 19 more
--------------------------------------
Specifying namenode works:
- oozie-setup sharelib create -fs hdfs://default-centos-66:8020 -locallib /usr/lib/oozie/oozie-sharelib-yarn
setting JAVA_LIBRARY_PATH="$JAVA_LIBRARY_PATH:/usr/lib/hadoop/lib/native"
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
setting OOZIE_HTTPS_PORT=11443
setting OOZIE_HTTPS_KEYSTORE_PASS=password
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.port=${OOZIE_HTTPS_PORT}"
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.keystore.pass=${OOZIE_HTTPS_KEYSTORE_PASS}"
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
setting JAVA_LIBRARY_PATH="$JAVA_LIBRARY_PATH:/usr/lib/hadoop/lib/native"
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
setting OOZIE_HTTPS_PORT=11443
setting OOZIE_HTTPS_KEYSTORE_PASS=password
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.port=${OOZIE_HTTPS_PORT}"
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.keystore.pass=${OOZIE_HTTPS_KEYSTORE_PASS}"
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/oozie/libtools/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/oozie/libserver/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
the destination path for sharelib is: /user/oozie/share/lib/lib_20150616133021