Uploaded image for project: 'Hue (READ ONLY)'
  1. Hue (READ ONLY)
  2. HUE-6834

[fb] Upgrade from boto 2.x to botocore to fix SignerType problems preventing alternative S3 endpoints from working

          Details

          • Type: Bug
          • Status: Closed
          • Priority: Major
          • Resolution: Incomplete
          • Affects Version/s: 3.12.0, 4.0.0
          • Fix Version/s: None
          • Component/s: app.filebrowser
          • Labels:

            Description

            Hue currently uses boto 2.46.1 however there are a few bugs in boto 2.x's handling of SignerTypes. We are interested in using Hue with different S3 end points and the SignerType bugs are a blocker.

            boto3 / botocore aren't affected by these bugs. @jennykim, do you think this is ok or is there a blocker preventing us from upgrading? The files that import boto are as follows:

            ./desktop/core/ext-py/django-extensions-1.5.0/django_extensions/management/commands/sync_s3.py
./desktop/core/ext-py/django-extensions-1.5.0/build/lib.linux-x86_64-2.7/django_extensions/management/commands/sync_s3.py
./desktop/libs/aws/src/aws/s3/s3_test.py
./desktop/libs/aws/src/aws/s3/s3file.py
./desktop/libs/aws/src/aws/s3/s3fs.py
./desktop/libs/aws/src/aws/s3/__init__.py
./desktop/libs/aws/src/aws/conf.py
./desktop/libs/aws/src/aws/client.py
./desktop/libs/aws/setup.py
./build/env/lib/python2.7/site-packages/django_extensions-1.5.0-py2.7.egg/django_extensions/management/commands/sync_s3.py

            The only risk I see is the django sync_s3 extension. Since boto 2.x and botocore use different package names, the django extension and the filebrowser could use both in the meantime. How do you feel about that change?

              Attachments

                Issue Links

                relates to

                Improvement - An improvement or enhancement to an existing feature or task. HUE-4894 [aws] Upgrade to boto 3

                • Critical - Crashes, loss of data, severe memory leak.
                • Closed

                  Activity

                    People

                    • Assignee:
                      Unassigned
                      Reporter:
                      ehiggs Ewan Higgs
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      2 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: