Uploaded image for project: 'RecordService (READ-ONLY)'
  1. RecordService (READ-ONLY)
  2. RS-195

Thread pool DrainAndShutdown() may never return under load

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.1.0
    • Fix Version/s: None
    • Component/s: Server
    • Labels:

      Description

      The util/thread-poolh defines the following method:

        /// Blocks until the work queue is empty, and then calls Shutdown to stop the worker
        /// threads and Join to wait until they are finished.
        /// Any work Offer()'ed during DrainAndShutdown may or may not be processed.
        void DrainAndShutdown() {
          {
            boost::unique_lock<boost::mutex> l(lock_);
            while (work_queue_.GetSize() != 0) {
              empty_cv_.wait(l);
            }
          }
          Shutdown();
          Join();
        }
      

      This method is called by RPC Close() method.

      Notice that it first waits for the queue to drain before shutting down. Nothing is preventing new requests from entering the queue, so it may block forever as long as there is a steady influx of new requests.

      Instead the code should prevent any new work from entering the pool and only then wait for existing ones to complete.

      Note that the same problem is present in Impala code as well.

        Attachments

          Activity

            People

            • Assignee:
              akolb Alex Kolbasov
              Reporter:
              akolb Alex Kolbasov
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: