Uploaded image for project: 'Livy (READ-ONLY)'
  1. Livy (READ-ONLY)
  2. LIVY-161

Livy internal configs show up in Spark History Server

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.2
    • Fix Version/s: 0.3
    • Component/s: RSC
    • Labels:
      None

      Description

      To pass Livy-specific options to the Spark driver, the RSC stashes them in the Spark configuration with the spark._livy_ prefix. The issue is that those show up in the "Environment" tab of the Spark UI and get written down to Spark event logs; so things like the secret used for authentication are visible.

      We need to remove those from the Spark config to be able to run Livy with security enabled.

        Attachments

          Activity

            People

            • Assignee:
              jerryshao Saisai Shao
              Reporter:
              vanzin Marcelo Vanzin
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: