Uploaded image for project: 'Hue'
  1. Hue
  2. HUE-2852

[hive] Support nested types autocomplete

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.8.0
    • Fix Version/s: 3.10.0
    • Component/s: app.hive, app.impala
    • Labels:
      None

      Description

      HUE-2768

      CREATE TABLE IF NOT EXISTS nested_test (
       a int,
       b ARRAY<STRUCT<c:INT, d:STRING>>,
       e MAP<STRING,INT>,
       f string,
       g ARRAY<INT>,
       h STRUCT<i:DOUBLE>
      ) PARTITIONED BY (p1 INT, p2 STRING)
      STORED AS PARQUET;
      
      INSERT OVERWRITE TABLE nested_test
      PARTITION (p1=1, p2="partition1")
      SELECT
        1,
        array(named_struct("c", 2, "d", "foo")),
        map("key1", 10, "key2", 20),
        "bar",
        array(2,3,4,5),
        named_struct("i", 1.23)
      FROM <sometable> limit 2;
      
      SELECT * FROM nested_test;
      SELECT a, exploded_col FROM nested_test LATERAL VIEW explode(g) exploded_tbl AS exploded_col;
      SELECT a, exploded_col.c, exploded_col.d FROM nested_test LATERAL VIEW explode(b) exploded_tbl AS exploded_col;
      SELECT a, exploded_key, exploded_value FROM nested_test LATERAL VIEW explode(e) exploded_tbl AS exploded_key, exploded_value;
      

      Syntax we could help with:
      array --> explode(c)
      struct --> c.f1
      array<struct> --> c.f1
      map --> explode(c) as (key, val)

      posexplode similar to explode but with pos
      posexplode(c) AS pos, myC

      Values we could help with:
      SELECT e['keys1']
      FROM nested_test;

      SELECT b[0]
      FROM nested_test;

      Google for hive complex types:
      https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-Built-inTable-GeneratingFunctions(UDTF)
      https://snehalatastechnotes.wordpress.com/2013/02/07/complex-data-types-in-hive/
      http://edu-kinect.com/blog/2014/06/16/hive-complex-data-types-with-examples/

      https://github.com/cloudera/hue/commit/4c02a423d9db667a9edf11914af1e2a7b6dd15a5

      Overall this seems the most useful to autocomplete. It would require pulling real data instead of just the types:

      1. struct a.b.c
      2. map['keys1']
      3. array[0]
      4. magic explode

      We can autocomplete in the WHERE clause.
      We can autocomplete values in WHERE clause in another jira.

      describe FORMATTED nested_Test.b.$elem$

        Attachments

          Activity

            People

            • Assignee:
              jennykim Jenny Kim
              Reporter:
              romain Romain Rigaux
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: