Details
Major Description
In CDH 4.7.0 in the commit c61ff8, we backported apps/oozie/src/oozie/migrations/0025_change_examples_path_format.py and renamed it to 0022_change_examples_path_format.py. When a user then upgrades their Hue instance to CDH 5.0.1+/Hue 3.5.0+, the upgrade process removes the 0022_change_examples_path_format.py file. This will result in the user receiving a ghost migration error like this:
Traceback (most recent call last): File "./build/env/bin/hue", line 9, in <module> load_entry_point('desktop==3.6.0', 'console_scripts', 'hue')() File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/desktop/core/src/desktop/manage_entry.py", line 60, in entry execute_manager(settings) File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/__init__.py", line 459, in execute_manager utility.execute() File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/__init__.py", line 382, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/base.py", line 196, in run_from_argv self.execute(*args, **options.__dict__) File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/base.py", line 232, in execute output = self.handle(*args, **options) File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/management/commands/migrate.py", line 111, in handle ignore_ghosts = ignore_ghosts, File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/migration/__init__.py", line 200, in migrate_app applied_all = check_migration_histories(applied_all, delete_ghosts, ignore_ghosts) File "/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/hue/build/env/lib/python2.6/site-packages/South-0.8.2-py2.6.egg/south/migration/__init__.py", line 95, in check_migration_histories raise exceptions.GhostMigrations(ghosts) south.exceptions.GhostMigrations: ! These migrations are in the database but not on disk: <oozie: 0022_change_examples_path_format> ! I'm not trusting myself; either fix this yourself by fiddling ! with the south_migrationhistory table, or pass --delete-ghost-migrations ! to South to have it delete ALL of these records (this may not be good).
This particular migration appears to be idempotent, so this gives us a couple options to fix this particular bug:
- We could just re-add the 0022_change_examples_path_format.py file to the Hue master branch, and all of the older branches. We added running our migrations with -
mergein 11582cbd, so this should be safe. However, always running with -merge could someday cause problems for a non-idempotent migration. - We could document that before you upgrade from 4.7.0 you should run hue migrate oozie 0021 to revert that migration.
- We could document that this particular migration should be deleted. This can be done with this script:
% ./build/env/bin/hue shell >>> from south.models import MigrationHistory >>> try: ... m = MigrationHistory.objects.filter(app_name='oozie', migration='0022_change_examples_path_format').get() ... except MigrationHistory.DoesNotExist: ... pass ... else: ... m.delete()