Details
Major Description
If you try to use grunt (the Pig shell) from the shell application with Kerberos enabled, you get the following error when running any commands that need to interact with HDFS:
2013-04-14 17:08:37,108 [main] ERROR org.apache.hadoop.security.UserGroupInformation - PriviledgedActionException as:joey (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 2013-04-14 17:08:37,112 [main] WARN org.apache.hadoop.ipc.Client - Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 2013-04-14 17:08:37,112 [main] ERROR org.apache.hadoop.security.UserGroupInformation - PriviledgedActionException as:joey (auth:KERBEROS) cause:java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
From testing, it looks like the issue is that we only obtain a delegation token for MapReduce and WebHDFS, not for regular HDFS. If I try to use the same delegation token file from the command line, the following fails in the same way:
hadoop fs -ls /
but this succeeds:
hadoop fs -ls webhdfs://<nn-host>:50070/
I think in order for shell commands that need to access HDFS to run, they'll need an HDFS delegation token as well.