Details
Description
When the Spark REPL is invoked, it sets its class loader as the currently active class loader in the thread, but there's no code to unset that.
What happens is that, since Livy uses the global execution context to invoke the REPL, all threads in that context end up referencing the REPL class loader, and thus references to it are never cleaned up.
In most cases this is fine since there REPL is running on its own separate VM; but this is bad for our unit tests since it causes SparkIMain instances to leak, and those reference a lot of memory.
Attachments
Issue Links
- links to