Uploaded image for project: 'Hue (READ ONLY)'
  1. Hue (READ ONLY)
  2. HUE-5768

[jb] Properly point to the correct node managers when fetching logs

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: 3.11.0
    • Fix Version/s: None
    • Component/s: app.jobbrowser
    • Labels:
      None

      Description

      Follow-up: HUE-3434

      Hi. There is some problems in fix provided for this issue.
      I'm reopening this bug, instead of creating one, because I can not clearly describe steps to reproduce.
      There is problem with getting logs for tasks attempts which was executed on host other than application master attempt.

      Problem in this piece of code (jobbrowser/yarn_models.py):

      class Attempt:
        ...
        def get_task_log(self, offset=0):
          logs = []
          attempt = self.task.job.job_attempts['jobAttempt'][-1]
          log_link = attempt['logsLink']
      
          # Generate actual task log link from logsLink url
          if self.task.job.status in ('NEW', 'SUBMITTED', 'RUNNING'):
            ...
          else:  # Completed jobs
            logs_path = '/jobhistory/logs/'
            root_url, tracking_path = log_link.split(logs_path)
            node_url, container_id, attempt_id, user = tracking_path.strip('/').split('/')
      
            # Replace log path tokens with actual attempt properties if available
            if hasattr(self, 'nodeHttpAddress') and 'nodeId' in attempt:
              node_url = '%s:%s' % (self.nodeHttpAddress.split(':')[0], attempt['nodeId'].split(':')[1])
      

      The problem is in node_url variable.
      You are trying to get nodeId for task attempt (which is host:port of NodeManager on that node), by getting nodeId for application master attempt and replacing hostname there. This is wrong way, because NodeManagers on different nodes have different ports (assigned randomly).

      Hope, that I described problem clearly

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jennykim Jenny Kim
                Reporter:
                romain Romain Rigaux
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: