Uploaded image for project: 'Flume (READ-ONLY)'
  1. Flume (READ-ONLY)
  2. FLUME-683

Add input/output buffering to ThriftEventSource/Sink

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: v0.9.4
    • Fix Version/s: v0.9.5
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      Some simple buffering should improve network performance quite a bit. I've so far implemented this only on the sending side (ThriftEventSink) and performance seems to have improved nicely, though I don't have good numbers for this yet. There are two possible solutions:

      1) Use the native Thrift TFastFramedTransport (buffered version of TFramedTransport); lazy socket changes will need to be ported over (for TSaneServerSocket)
      2) Wrap the existing code with a buffered version (see TBufferedTransport in patch)

      I used the latter method for a simple test implementation only on the sending (sink) side.

        Attachments

          Activity

            People

            • Assignee:
              chetan Chetan Sarva
              Reporter:
              csarva Chetan Sarva
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: