Uploaded image for project: 'CDH (READ-ONLY)'
  1. CDH (READ-ONLY)
  2. DISTRO-847

Hive select from tables with join don't give results depend on rows count

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: CDH 5.8.3
    • Fix Version/s: None
    • Component/s: Hive
    • Labels:
    • Environment:
      Really this issue happens on CDH 5.10.x

      Description

      Hi Community,

      We have a script - collmx_consents_snp.hql (in attachement).
      In this script we use join of following tables:

      consent_service_consent_hst
      consent_service_consent_subject_hst
      consent_service_client_hst

      DDL of those table are in attachment too.

      All of this tables are partitioned by date.

      The root cause of problem is that join of two tables in production don't working(there is consistent data in tables but query don't get nothing):
      select *
      from (
      SELECT consent_uid,CASE WHEN for_contract = true THEN evid_srv ELSE NULL END evid_srv,entity_type,to_date(modif_time) apply_date,id_client
      FROM consent_service_consent_snp
      ) csc
      join consent_service_consent_subject_snp cscs on (csc.consent_uid = cscs.consent_uid)
      In test environment all fine.

      1.png
      When we add some filter on table consent_service_consent_snp by partition, than query giva us results:
      2.jpg
      When we are running select count on this tables, we didn't get any errors. In our test environment we have less data than in production. And when we add constraint on date in select clause all working fine, so we think that problem may depend on number of rows in the table. Logs of HiveServer2 and HiveMetastore in attachement.
      When query is fails we see following in hiveserver2 log:
      2017-03-09 20:00:30,406 INFO org.apache.hadoop.hive.ql.plan.ConditionalResolverCommonJoin: [HiveServer2-Background-Pool: Thread-7269]: Failed to resolve driver alias (threshold : 25000000, length mapping :

      {cscs:consent_service_consent_subject_hst=571829172, csc:consent_service_consent_snp:consent_service_consent_hst=434475747}

      )

        Attachments

        1. 1.png
          1.png
          25 kB
        2. 2.jpg
          2.jpg
          76 kB
        3. all attachements.rar
          5.09 MB

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              rakhmadeev Akhmadeev Ramil
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: