Details
Description
Hi Community,
We have a script - collmx_consents_snp.hql (in attachement).
In this script we use join of following tables:
consent_service_consent_hst
consent_service_consent_subject_hst
consent_service_client_hst
DDL of those table are in attachment too.
All of this tables are partitioned by date.
The root cause of problem is that join of two tables in production don't working(there is consistent data in tables but query don't get nothing):
select *
from (
SELECT consent_uid,CASE WHEN for_contract = true THEN evid_srv ELSE NULL END evid_srv,entity_type,to_date(modif_time) apply_date,id_client
FROM consent_service_consent_snp
) csc
join consent_service_consent_subject_snp cscs on (csc.consent_uid = cscs.consent_uid)
In test environment all fine.
1.png
When we add some filter on table consent_service_consent_snp by partition, than query giva us results:
2.jpg
When we are running select count on this tables, we didn't get any errors. In our test environment we have less data than in production. And when we add constraint on date in select clause all working fine, so we think that problem may depend on number of rows in the table. Logs of HiveServer2 and HiveMetastore in attachement.
When query is fails we see following in hiveserver2 log:
2017-03-09 20:00:30,406 INFO org.apache.hadoop.hive.ql.plan.ConditionalResolverCommonJoin: [HiveServer2-Background-Pool: Thread-7269]: Failed to resolve driver alias (threshold : 25000000, length mapping :
)