Details
-
Type:
Improvement
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: v0.9.4
-
Component/s: Sinks+Sources
-
Labels:None
Description
Initiated by Todd:
"I think the current syntaxes being discussed are quite complicated. As Jon and I have discussed previously, Flume really needs support for some kind of keyword arguments before we can really make a complex sink like this usable."
The syntax to implement is yet to be defined. I'm for Todd's variant:
hbaseSink('table', 'mytable', 'buffering', 'true', 'sendBody', 'true', 'mapping.body',
'mycf:myqual', 'mapping.attr.exampleAttr', 'mycf:example', ...).
Major reasons:
a) it will be easier to add new params while preserving compatibility with elder configurations
b) sink has many optional parameters which if we go "order-sticky" way should be all provided explicitly (or with "nulls")
I also suggest to use attr2hbase sink as a base for the sink with improved syntax as it contains many shared logic for HBase sinks.