Uploaded image for project: 'Hue'
  1. Hue
  2. HUE-8677

[oozie] Oozie batch snippet hangs

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 4.3.0
    • Fix Version/s: None
    • Component/s: app.jobbrowser
    • Labels:
      None
    • Environment:

      Oozie 5.1.0

      Description

      For example, I'm trying to execute following code in Sqoop1 editor:

      import --connect jdbc:mysql://10.2.14.185:3306/hue --table auth_user --username root --password 123456 --target-dir /tmp/out_46
      

      After some time, I can see that Oozie job is finised by using Oozie CLI, while in Hue Editor interface job is still in "running state" and I see following exception in Hue logs continuously:

      [17/Dec/2018 09:25:33 +0000] connectionpool DEBUG    "node1.cluster.com:8088 GET /ws/v1/cluster/apps/application_1544708734813_0043 HTTP/1.1" 200 None
      [17/Dec/2018 09:25:33 +0000] resource     DEBUG    GET /cluster/apps/application_1544708734813_0043 Got response in 14ms: {"app":{"id":"application_1544708734813_0043","user":"ivan","name":"oozie:launcher:T=sqoop:W=Batch job for query-sqoop1:A=sqoop-0cab:ID=0000000-181217155734702-oozie-ivan-W","queue":"root.ivan","state":"FINISHED","finalStatus":"SUCCEEDED","progress":100.0,"trackingUI":"History","trackingUrl":"http://node1.cluster.com:8088/proxy/application_1544708734813_0043/","diagnostics":"","clusterId":1544708734813,"applicationType":"Oozie Launcher","applicationTags":"","startedTime":1545062632830,"finishedTime":1545062670674,"elapsedTime":37844,"amContainerLogs":"http://node2.cluster.com:8042/node/containerlogs/container_e02_1544708734813_0043_01_000001/ivan","amHostHttpAddress":"node2.cluster.com:8042","allocatedMB":-1,"allocatedVCores":-1,"runningContainers":-1,"memorySeconds":80572,"vcoreSeconds":39,"preemptedResourceMB":0,"preemptedResourceVCores":0,"numNonAMContainerPreempted":0,"numAMContainerPreempted":0}}
      [17/Dec/2018 09:25:33 +0000] connectionpool DEBUG    "node1.cluster.com:8088 GET /ws/v1/cluster/apps/application_1544708734813_0043/appattempts HTTP/1.1" 200 None
      [17/Dec/2018 09:25:33 +0000] resource     DEBUG    GET /cluster/apps/application_1544708734813_0043/appattempts Got response in 17ms: {"appAttempts":{"appAttempt":[{"id":1,"startTime":1545062632900,"containerId":"container_e02_1544708734813_0043_01_000001","nodeHttpAddress":"node2.cluster.com:8042","nodeId":"node2.cluster.com:8099","logsLink":"//node2.cluster.com:8042/node/containerlogs/container_e02_1544708734813_0043_01_000001/ivan"}]}}
      [17/Dec/2018 09:25:33 +0000] api          ERROR    An error occurred while watching the job running
      Traceback (most recent call last):
        File "/home/ivan/hue/hue/apps/oozie/src/oozie/views/api.py", line 440, in get_log
          data = job_single_logs(request, **{'job': action.externalId})
        File "/home/ivan/hue/hue/apps/jobbrowser/src/jobbrowser/views.py", line 85, in decorate
          return view_func(request, *args, **kwargs)
        File "/home/ivan/hue/hue/apps/jobbrowser/src/jobbrowser/views.py", line 408, in job_single_logs
          failed_tasks = job.filter_tasks(task_states=('failed',))
        File "/home/ivan/hue/hue/apps/jobbrowser/src/jobbrowser/yarn_models.py", line 400, in filter_tasks
          return [self.get_task(0)]
        File "/home/ivan/hue/hue/apps/jobbrowser/src/jobbrowser/yarn_models.py", line 396, in get_task
          task.taskAttemptIds = [appAttempt['appAttemptId'] for appAttempt in self.job_attempts['jobAttempt']]
        File "/home/ivan/hue/hue/apps/jobbrowser/src/jobbrowser/yarn_models.py", line 407, in job_attempts
          attempt['id'] = attempt['appAttemptId']
      KeyError: 'appAttemptId'
      

        Attachments

          Activity

            People

            • Assignee:
              vanyasmart Ivan Dzikovsky
              Reporter:
              vanyasmart Ivan Dzikovsky
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: