Uploaded image for project: 'CDH (READ-ONLY)'
  1. CDH (READ-ONLY)
  2. DISTRO-464

Sqoop: Importing the updated records into Hive using incremental import.

          Details

          • Type: New Feature
          • Status: Closed
          • Priority: Major
          • Resolution: Not A Bug
          • Affects Version/s: CDH4.1.3
          • Fix Version/s: None
          • Component/s: Sqoop
          • Labels:
          • Environment:
            Cloudera Hadoop - CDH-4.1.3
            Sqoop - 1.4.1
            Hive - 0.9.0
            Java - 1.6

            Description

            We are not able to import Updated records (from MySql) into Hive using incremental import. The following happens in the incremental modes:
            append Mode - Its not able to fetch the record itself.
            lastModified Mode - Its duplicating records into Hive.

            It would be better to have command which intimates or finds the updated records in the table based on Timestamp column and replace existing records in Hive with the updated records.

            Note: Updated records doesn't mean that newly inserted records.

              Attachments

                Activity

                  People

                  • Assignee:
                    Unassigned
                    Reporter:
                    sureeceg Suresh Srinivasan
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    2 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved: