Details
-
Type:
Improvement
-
Status: Open
-
Priority:
Minor
-
Resolution: Unresolved
-
Affects Version/s: 0.2
-
Fix Version/s: None
-
Component/s: API
-
Labels:None
Description
Exposing YARN ID and YARN logs
Exposing YARN ID and YARN logs will improve the manageability of Livy. As they are YARN specific, they should be separated to a new nested interface.
I think we can add nested resource cluster in batch & session to expose these cluster manager specific APIs.
Example response:
GET /sessions/{{id}}/cluster
{
"type": "yarn",
"applicationId": "application_xx_yy_zz"
}
GET /sessions/{{id}}/cluster/log
{
"from":..,
"total":..,
"log":[]
}
Optionally, we can automatically return cluster specific information in the {{batch}} & {{session}} objects:
GET /sessions/{{id}}
{
id:..,
state:..
kind:..,
log: []
cluster: {
"type": "yarn",
"applicationId": "application_xx_yy_zz"
}
}
Interface definitions:
interface Cluster {
val type: String
}
interface YarnCluster extends Cluster {
val applicationId: String
def log(): String
}
applicationId and log APIs will only be available when cluster.type == "yarn"