Details
Description
It seems the only correct way to run Livy server is to run it under hue user in a kerberized cluster?
There are two advantages:
1. hue user is normally (by default in CDH) is allowed to impersonate to other users;
2. hue already has a keytab maintened by Cloduera Manager.
This is explained in http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/cm_sg_yarn_long_jobs.html
"Configuring Spark on YARN for Long-running Applications".
Assuming Livy Server creates long-running Spark applications.
Without --principal and --keytab parameters (and without non-expired kerberos ticket), I am getting:
The Spark session could not be created in the cluster: ... 37 more
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) ...
46 more
Without --principal and --keytab parameters, but with a non-expired kerberos ticket for my personal principal, I am getting:
16/01/16 22:59:00 INFO RMProxy:
Connecting to ResourceManager at xx.yy.com/10.20.xx.yy:8032
16/01/16 22:59:00 WARN UserGroupInformation:
PriviledgedActionException as:rdautkha (auth:PROXY)
via rdautkhanov@XX.COM (auth:KERBEROS)
cause:org.apache.hadoop.security.authorize.AuthorizationException:
User: rdautkhanov@XX.COM is not allowed to impersonate rdautkha
ps. hue.keytab is located in
$ sudo ls -ltr /var/run/cloudera-scm-agent/process | grep hue-KT_RENEWER | tail -1 | awk '
{print $9}'
on hosts running "Kerberos Ticket Renewer" Hue role. This is the directory that must be passed to --keytab parameter to spark-submit.
Irrespective of Hue, Livy Server should have parameters to allow to pass --principal and --keytab to spark-submit. So it's easier (or maybe the only way) to use in kerberized environmens.
Attachments
Issue Links
- depends on
-
LIVY-3 Add Security
- Open