Uploaded image for project: 'Hue'
  1. Hue
  2. HUE-2447

desktop migration 0007 should check if the tables already exist

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.5.0, 3.6.0, 3.7.0
    • Fix Version/s: None
    • Component/s: core.backend
    • Labels:
      None

      Description

      In the migration 0007_auto__add_documentpermission__add_documenttag__add_document.py, the Document.objects.sync() on this line line is particularly expensive. And so it's possible a user might interrupt the migration while it's occurring. If this happens, and the user is using a database that doesn't support atomic DDL transactions like MySQL or Oracle, it's possible the next time the migration is run some of the tables will have already been created and error out with this exception:

      Traceback (most recent call last):
        File "./build/env/bin/hue", line 9, in <module>
          load_entry_point('desktop==3.6.0', 'console_scripts', 'hue')()
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/desktop/core/src/desktop/manage_entry.py", line 60, in entry
          execute_manager(settings)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/__init__.py", line 459, in execute_manager
          utility.execute()
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/__init__.py", line 382, in execute
          self.fetch_command(subcommand).run_from_argv(self.argv)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/base.py", line 196, in run_from_argv
          self.execute(*args, **options.__dict__)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/base.py", line 232, in execute
          output = self.handle(*args, **options)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/management/commands/migrate.py", line 111, in handle
          ignore_ghosts = ignore_ghosts,
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/migration/__init__.py", line 220, in migrate_app
          success = migrator.migrate_many(target, workplan, database)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/migration/migrators.py", line 229, in migrate_many
          result = migrator.__class__.migrate_many(migrator, target, migrations, database)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/migration/migrators.py", line 304, in migrate_many
          result = self.migrate(migration, database)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/migration/migrators.py", line 129, in migrate
          result = self.run(migration, database)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/migration/migrators.py", line 113, in run
          return self.run_migration(migration, database)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/migration/migrators.py", line 83, in run_migration
          migration_function()
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/migration/migrators.py", line 59, in <lambda>
          return (lambda: direction(orm))
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/desktop/core/src/desktop/migrations/0007_auto__add_documentpermission__add_documenttag__add_document.py", line 17, in forwards
          ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/db/generic.py", line 47, in _cache_clear
          return func(self, table, *args, **opts)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/db/generic.py", line 361, in create_table
          "columns": ', '.join([col for col in columns if col]),
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/db/generic.py", line 282, in execute
          cursor.execute(sql, params)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/db/backends/mysql/base.py", line 114, in execute
          return self.cursor.execute(query, args)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-linux-x86_64.egg/MySQLdb/cursors.py", line 173, in execute
          self.errorhandler(self, exc, value)
        File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-linux-x86_64.egg/MySQLdb/connections.py", line 36, in defaulterrorhandler
          raise errorclass, errorvalue
      django.db.utils.DatabaseError: (1050, "Table 'desktop_documentpermission' already exists")
      

      The simplest thing that we could do to mitigate this migration bug is to modify this script to check if the table already exists before we try to create it, as we do in 0001_permissions_and_profiles.py.

        Attachments

          Activity

            People

            • Assignee:
              erickt Erick Tryzelaar
              Reporter:
              erickt Erick Tryzelaar
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: