Now that Flume has a DatasetSink, it should be possible to use Flume's log4j appender rather than our own.
Switch examples to use DatasetSink and Flume's log4j appender
Flume layout partitioning should be configured automatically
Automatically set flume path from the partition strategy
Add flume DatasetSink