Uploaded image for project: 'Hue (READ ONLY)'
  1. Hue (READ ONLY)
  2. HUE-2716

[pig] Scripts fail on hcat auth with org.apache.hive.hcatalog.pig.HCatLoader()


    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.8.0
    • Fix Version/s: 3.9.0
    • Component/s: con.pig
    • Labels:


      If a Hue Pig script uses org.apache.hive.hcatalog.pig.HCatLoader(), it will currently fail with hcat auth errors.

      In: apps/pig/src/pig/models.py use_hcatalog(), it will set the hcat credentials in the submitted Pig script workflow if it finds "org.apache.hcatalog.pig.HCatLoader", but it is not looking for "org.apache.hive.hcatalog.pig.HCatLoader". When the users uses the latter in a script, the hcat creds don't get included, and the pig script and workflow will fail to work.

      "org.apache.hcatalog.pig.HCatLoader" will be deprecated very soon in Hcat, so we need to move to compatibility with "org.apache.hive.hcatalog.pig.HCatLoader".

      We found this in Hue 3.7 but it appears to still be an issue on Hue master (and thus also in 3.8).

      It is very easy to fix as you only need to update the return from use_catalog() to consider the new APIs, and can maintain backward compatibility with the old ones as well. This is what we used and tested to fix in our Hue deployment:

      return ('org.apache.hcatalog.pig.HCatStorer' in script or 'org.apache.hcatalog.pig.HCatLoader' in script) or \
      ('org.apache.hive.hcatalog.pig.HCatLoader' in script or 'org.apache.hive.hcatalog.pig.HCatStorer' in script)




            • Assignee:
              rickbernotas Rick Bernotas
              rickbernotas Rick Bernotas
            • Votes:
              0 Vote for this issue
              2 Start watching this issue


              • Created: