Uploaded image for project: 'Hue'
  1. Hue
  2. HUE-1040

${nameNode} is added in dataset paths, and also added in workflows' <prepare> => double namenode address

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.2.0
    • Component/s: con.oozie
    • Labels:
    • Environment:

      Description

      I set an output dataset to

      /output-yz/${YEAR}${MONTH}${DAY}${HOUR}"

      The XML definition becomes:

      <datasets>
        <dataset name="output-01" frequency="${coord:days(1)}"
                 initial-instance="2013-02-11T21:00Z" timezone="GMT">
          <uri-template>${nameNode}/output-yz/${YEAR}${MONTH}${DAY}${HOUR}</uri-template>
          <done-flag>_SUCCESS</done-flag>
        </dataset>
      </datasets>
      <output-events>
        <data-out name="output" dataset="output-01">
        <instance>${coord:current(0)}</instance>
        </data-out>
      </output-events>
      

      In the workflow, the first action has a Prepare/delete of ${output}.

      Excerpt of the generated workflow's XML definition:

      <prepare>
          <delete path="${nameNode}${output}"/>
      </prepare>
      

      When the coordinator runs the workflow, I get this error:

      Heart beat
      Starting the execution of prepare actions
      Prepare execution in the Launcher Mapper has failed
      Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exception invoking main(), Incomplete HDFS URI, no host: hdfs://myhdfshost:8020hdfs:/myhdfshost:8020/output-yz/2013021121
      org.apache.oozie.action.hadoop.LauncherException: Incomplete HDFS URI, no host: hdfs://myhdfshost:8020hdfs:/myhdfshost:8020/output-yz/2013021121
      

      3.2.4 shows that the path in <delete> does not have to include the namenode address.

      HUE-982 says that dataset paths must include the namenode address, so I suggest that ${nameNode} should not be automatically added to the Prepare/delete value.

        Attachments

          Activity

            People

            • Assignee:
              romain Romain Rigaux
              Reporter:
              nfo Nicolas Fouché
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: