When the MapReduce launcher job stops, Yarn moves the job to the History server. If we did not fetch the last part of the logs, we are going to miss it unless the user refresh the page after a few seconds.
We should revert https://github.com/cloudera/hue/commit/739dff7cb4e5fc23b4147386a0df0edf9abc5a2c and stop the fetching of the log when we saw the '<<< Invocation of Pig command completed <<<' in the logs (note, we don't display this message). Beware that a user could restart the job or another one so we need to display the Pig script as finished only when workflow status is finished + last log line was fetched. This will provide a better UX too.
We can chat more if needed