Reliable Flume log processing with Oozie

The demo example uses an Oozie trick to process the previous Flume partition, but this is not reliable since i) partitions may be empty (in which case the next partition is never processed), and ii) events can come in after the partition has been processed (in which case they are missed, since the partition is never regenerated).

Lance Riedel pointed me to this Oozie user thread about the problem: http://mail-archives.apache.org/mod_mbox/oozie-user/201305.mbox/%3C5D73AAAC-1069-49C0-AE95-FD87A20692A3@hivedata.com%3E

This JIRA is to discuss better approaches.