Uploaded image for project: 'Flume (READ-ONLY)'
  1. Flume (READ-ONLY)
  2. FLUME-635

Allow the compression codec to be specified in SequenceFiles

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v0.9.4
    • Component/s: None
    • Labels:
      None

      Description

      The dfs sink writes sequence files but doesn't honour flume.collector.dfs.compress.codec. On the other hand, the customdfs sink supports compression (via flume.collector.dfs.compress.codec), but doesn't write SequenceFiles.

      This can be fixed by either changing the dfs sink to honour flume.collector.dfs.compress.codec (compression would be enabled by setting io.seqfile.compression.type), or by creating a SequenceFileOutputFormat for use with the customdfs sink.

        Attachments

        1. FLUME-635-1.patch
          7 kB
          Tom White
        2. FLUME-635-2.patch
          14 kB
          Tom White
        3. FLUME-635-2.patch
          14 kB
          Tom White

          Issue Links

            Activity

              People

              • Assignee:
                tom Tom White
                Reporter:
                tom Tom White
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: