Uploaded image for project: 'RecordService (READ-ONLY)'
  1. RecordService (READ-ONLY)
  2. RS-19 S3 Support
  3. RS-164

S3 takes a long time to load block locations

          Details

          • Type: Sub-task
          • Status: Open
          • Priority: Major
          • Resolution: Unresolved
          • Affects Version/s: 0.3.0
          • Fix Version/s: None
          • Component/s: Server
          • Labels:

            Description

            For tables with large number of partitions, it takes a long time to load the partitions:

            I0420 22:28:11.121326 42029 HdfsTable.java:508] LOAD_BLOCK_LOCATION.tpcds500gb_parquet_s3.store_sales: 60189millisec

            This happens to both Impala and RS, but because Impala has metadata cache, it's not a big issue.
            We need to investigate why it is so slow.

              Attachments

                Activity

                  People

                  • Assignee:
                    Unassigned
                    Reporter:
                    sunchao Chao Sun
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    3 Start watching this issue

                    Dates

                    • Created:
                      Updated: