Details
-
Type:
Sub-task
-
Status: Resolved
-
Priority:
Major
-
Resolution: Won't Fix
-
Affects Version/s: 3.8.0
-
Fix Version/s: None
-
Component/s: con.spark
-
Labels:None
Description
It would be nice if livy could reuse an Application Master to speed up it's startup time. However this is non-trivial to do as we're letting Spark manage getting resources from YARN. We'd have to write our own Application Master and proxy requests to and from the spark context in order for it to allocate it's nodes.