Uploaded image for project: 'Flume (READ-ONLY)'
  1. Flume (READ-ONLY)
  2. FLUME-155

Formalise behaviour of sinks, sources and decorators

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v0.9.1u1, v0.9.2
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      Although sinks, sources et al have pretty simple APIs, the state diagram for how they should behave in response to these methods is left undefined.

      For example:

      1. Can open(..) block?
      2. What should the behaviour be if open(...) is called twice in succession without an intervening close?
      3. What if next(..) is called after close(..)? or before open(...)?
      4. What should the policy be if a source has 'pending' events and close(..) is called?

      There are many more similar questions. Some of them are already answered by convention in the code, where others are left unspecified. It should be pretty easy to formalise the answers, maybe via a state machine diagram (where the transitions are method calls).

      There have been a lot of bugs inside Flume due to misbehaving sources or sinks, but given no formal specification to adhere to there was no way they could have been known to be buggy. Once we figure out the exact semantics of each call, we can write a test suite that puts every source and sink through its paces, and validate its behaviour, making it easier to find bugs.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jon Jonathan Hsieh
                Reporter:
                henry Henry Robinson
              • Votes:
                2 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: