Uploaded image for project: 'Hue'
  1. Hue
  2. HUE-2880

[hadoop] Failure when uploading large files to a kerberized HTTPFS

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.8.0
    • Fix Version/s: 3.9.0
    • Component/s: core.backend
    • Labels:
      None

      Description

      Hue is unable to upload large files into a kerberized cluster when it is configured to talk to a HDFS HTTPFS server. Here is what's going on:

      • Hue's webhdfs creates a session that preserves cookies.
      • On Hue's first access to HTTPFS, it makes a request in which HTTPFS returns a 401. Hue then gets a kerberos ticket and passes it along to talk to HTTPFS, which also returns a cookie in the response.
      • When Hue uploads a file, it first makes an empty request in order to get redirected to the actual place to perform the upload. It also passes along the cookies it received previously.
      • Finally it uploads the file. This time however it creates a new session without the cookies. Normally we would just go through the standard kerberos authentication process (401 and a retry), but if the file is over a certain size, it outright rejects the upload and abruptly just closes the connection in order to protect against malicious users.
      • Hue then displays an Error: Undefined popup because of the connection error not being properly rendered in the UI.

      The fix is simple, just make sure to reuse the session when uploading the file.

        Attachments

          Activity

            People

            • Assignee:
              erickt Erick Tryzelaar
              Reporter:
              erickt Erick Tryzelaar
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: