Details
Description
Currently, Livy redirects the spark log to $LIVY_HOME//loglivy-livy-server.out log file for Interactive Sessions.
So, when a session fails before initiating the SparkContext, then user doesn't have a way to know why his job got failed unless he is going to $LIVY_HOME//loglivy-livy-server.out to debug. But, all users are not going to have access to that log file.
Hence, it would be better if we redirect the spark log lines to REST response log field for debugging (as how we are doing for Batch session).