Uploaded image for project: 'Hue (READ ONLY)'
  1. Hue (READ ONLY)
  2. HUE-2716

[pig] Scripts fail on hcat auth with org.apache.hive.hcatalog.pig.HCatLoader()

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.8.0
    • Fix Version/s: 3.9.0
    • Component/s: con.pig
    • Labels:
      None

      Description

      If a Hue Pig script uses org.apache.hive.hcatalog.pig.HCatLoader(), it will currently fail with hcat auth errors.

      In: apps/pig/src/pig/models.py use_hcatalog(), it will set the hcat credentials in the submitted Pig script workflow if it finds "org.apache.hcatalog.pig.HCatLoader", but it is not looking for "org.apache.hive.hcatalog.pig.HCatLoader". When the users uses the latter in a script, the hcat creds don't get included, and the pig script and workflow will fail to work.

      "org.apache.hcatalog.pig.HCatLoader" will be deprecated very soon in Hcat, so we need to move to compatibility with "org.apache.hive.hcatalog.pig.HCatLoader".

      We found this in Hue 3.7 but it appears to still be an issue on Hue master (and thus also in 3.8).

      It is very easy to fix as you only need to update the return from use_catalog() to consider the new APIs, and can maintain backward compatibility with the old ones as well. This is what we used and tested to fix in our Hue deployment:

      return ('org.apache.hcatalog.pig.HCatStorer' in script or 'org.apache.hcatalog.pig.HCatLoader' in script) or \
      ('org.apache.hive.hcatalog.pig.HCatLoader' in script or 'org.apache.hive.hcatalog.pig.HCatStorer' in script)

        Attachments

          Activity

            People

            • Assignee:
              rickbernotas Rick Bernotas
              Reporter:
              rickbernotas Rick Bernotas
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: