Uploaded image for project: 'Hue (READ ONLY)'
  1. Hue (READ ONLY)
  2. HUE-2142

[core] Automated scalable download of query results

          Details

          • Type: Improvement
          • Status: Closed
          • Priority: Critical
          • Resolution: Incomplete
          • Affects Version/s: 3.6.0, 4.3.0
          • Fix Version/s: None
          • Component/s: core.api
          • Labels:
            None

            Description

            A Webserver:

            1. cannot serve/convert a lot of data
            2. cannot pull the query statuses continuously and download / cache the first part of results before releasing the query resources
            3. resultsetl conversion takes minutes to be done and give a "hanging" feeling to the user

            We probably need a task server to free it and scale downloads.

            Note: supporting XLS output too would be helpful

            FI: Beelines has these options https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-BeelineCommandOptions

            Priority is on SQL (Hive, Impala).

            With optionally

            1. Any Editors
            2. Search
            3. HDFS

              Attachments

                Issue Links

                depends on

                New Feature - A new feature of the product, which has yet to be developed. HUE-8259 [core] Task server libs

                • Major - Major loss of function.
                • Resolved
                is parent of

                Bug - A problem which impairs or prevents the functions of the product. HUE-8747 [editor] Download query result as task

                • Major - Major loss of function.
                • Resolved
                relates to

                Sub-task - The sub-task of the issue HUE-2527 [beeswax] XLS downloads with large column/row counts

                • Critical - Crashes, loss of data, severe memory leak.
                • Resolved

                Sub-task - The sub-task of the issue HUE-2256 [beeswax] Speed up deserialization API

                • Major - Major loss of function.
                • Resolved

                Sub-task - The sub-task of the issue HUE-4181 [hive] API to provide result set row count and data size

                • Major - Major loss of function.
                • Resolved

                Bug - A problem which impairs or prevents the functions of the product. HUE-2088 High CPU usage and slowness on downloads

                • Blocker - Blocks development and/or testing work, production could not run.
                • Resolved

                Bug - A problem which impairs or prevents the functions of the product. HUE-4201 [editor] Add max limit of rows before truncation in the export / download query result

                • Major - Major loss of function.
                • Resolved

                Improvement - An improvement or enhancement to an existing feature or task. HUE-2523 [core] Add support for xlsx export

                • Major - Major loss of function.
                • Resolved

                Bug - A problem which impairs or prevents the functions of the product. HUE-3871 [editor] Save large file as excel, didn’t even start

                • Major - Major loss of function.
                • Resolved

                New Feature - A new feature of the product, which has yet to be developed. HUE-6766 [search] Support scalable download of millions of results

                • Critical - Crashes, loss of data, severe memory leak.
                • Closed

                Improvement - An improvement or enhancement to an existing feature or task. HUE-3096 [core] Warn the user when query results exprted to CSV/XLS are truncated

                • Major - Major loss of function.
                • Resolved

                Improvement - An improvement or enhancement to an existing feature or task. HUE-2244 Add an option to select the number of rows in an csv file have to be skipped before rows are considered to be data rows.

                • Major - Major loss of function.
                • Closed

                  Activity

                    People

                    • Assignee:
                      jgauthier Jean Francois Desjeans Gauthier
                      Reporter:
                      romain Romain Rigaux
                    • Votes:
                      2 Vote for this issue
                      Watchers:
                      8 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: