Uploaded image for project: 'Livy (READ-ONLY)'
  1. Livy (READ-ONLY)
  2. LIVY-248

Clean up how PySpark python intepreters are set.

          Details

          • Type: Improvement
          • Status: Resolved
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: 0.3
          • Fix Version/s: 0.3
          • Component/s: Interpreter
          • Labels:
            None

            Description

            Right now they are set using env `PYSPARK3_DRIVER_PYTHON` and `PYSPARK_DRIVER_PYTHON`.

            Firstly, they should be called `PYSPARK3_PYTHON` and `PYSPARK_PYTHON`, because we are passing them to PYSPARK_PYTHON. Executors will use this python too, not just the driver.

            We should also document pyspark kind in README.rst.

              Attachments

                Activity

                  People

                  • Assignee:
                    tc0312 Alex Man
                    Reporter:
                    tc0312 Alex Man
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    1 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved: