Uploaded image for project: 'Livy (READ-ONLY)'
  1. Livy (READ-ONLY)
  2. LIVY-155

SparkInterpreter causes SparkIMain instances to leak

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.2
    • Fix Version/s: 0.2, 0.3
    • Component/s: REPL
    • Labels:
      None

      Description

      When the Spark REPL is invoked, it sets its class loader as the currently active class loader in the thread, but there's no code to unset that.

      What happens is that, since Livy uses the global execution context to invoke the REPL, all threads in that context end up referencing the REPL class loader, and thus references to it are never cleaned up.

      In most cases this is fine since there REPL is running on its own separate VM; but this is bad for our unit tests since it causes SparkIMain instances to leak, and those reference a lot of memory.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vanzin Marcelo Vanzin
                Reporter:
                vanzin Marcelo Vanzin
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: