Details
-
Type:
New Feature
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 0.15.0
-
Fix Version/s: None
-
Component/s: Data Module
Description
When writing datasets destined analysis with Impala, it would be helpful to be able to automate invalidating metadata for new tables & refreshing metadata for new data in existing tables.
For example, when you declare your dataset writer, you could add some metadata saying that this dataset is intended for Impala and once you close or flush, automatically issue the refresh command. Or, run the invalidate metadata when creating a new dataset.