Details
-
Type:
Improvement
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 0.3
-
Fix Version/s: None
-
Component/s: Core
-
Labels:None
Description
Livy Server when launches a Spark Job on yarn-cluster mode, RpcServer binds a random port and ContextLauncher sets that as livy.rsc.launcher.port. The RSCDriver which runs on some NodeManager Machine will try to connect to livy server at livy.rsc.launcher.port. In cluster where only fixed ports are open for security purposes, this job will fail. The livy.rsc.launcher.port port has to be a configurable one with a single or range of ports which RpcServer has to bind.