Uploaded image for project: 'Flume (READ-ONLY)'
  1. Flume (READ-ONLY)
  2. FLUME-431

Add attribute with dir path to events produced by tailing sources

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      Currently tailing sources (tail, taildir) add attribute with filename to event. Looks like we should put dir names too, since this can be critical info for event's processing logic.

      E.g. tailing collectD output - it outputs stats data into dirs like /ubuntu904desktop.localdomain/cpu-0, /ubuntu904desktop.localdomain/cpu-1, etc. Here, path info is important for logs processing logic and should be available too.

      Jon's suggestion:

      My suggestion is to use a kwarg to make recording the dir a open, and to give the user option to pick attribute name.

      So something like this:
      tailDir("/dir",".*", 0, dirAttr="tailfiledir", fileAttr="tailfilename")

      So a file /dir/file would produce events with metadata like:
      event

      { taildirfile="/dir", tailfilename="file" }

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                alex Alex Baranau
                Reporter:
                alex Alex Baranau
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: