Details
-
Type: Sub-task
-
Status: Open
-
Priority: Major
-
Resolution: Unresolved
-
Affects Version/s: 0.1.0
-
Fix Version/s: None
-
Component/s: Data Module
-
Labels:None
Description
The idea here is that if you have dataset partitioned by two fields: app_id and timestamp:
/events/app_id=1/timestamp=0/<files>
/events/app_id=1/timestamp=1/<files>
...
/events/app_id=1/timestamp=999/<files>
/events/app_id=2/timestamp=0/<files>
you should be able to ask for a subpartition (the user may not care about the fact is it a subpartition) timestamp=0:
/events/app_id=1/timestamp=0/<files>
/events/app_id=2/timestamp=0/<files>
The partition key in this case is (null, 0).