Uploaded image for project: 'CDH (READ-ONLY)'
  1. CDH (READ-ONLY)
  2. DISTRO-337

Upgrade from cdh3u0 to cdh3u1 breaks tasktracker log file ownership.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: CDH3u1
    • Fix Version/s: CDH3u4
    • Component/s: MapReduce, Packaging
    • Labels:
      None

      Description

      Following the upgrade procedures here:

      https://ccp.cloudera.com/display/CDHDOC/Upgrading+to+CDH3

      When I execute "sudo yum install hadoop-0.20", the ownership on all the /var/log/hadoop-hadoop-tasktracker* files changes from 'mapred mapred' or 'mapred root' to 'hdfs hadoop'.

      before "sudo yum install hadoop-0.20":

      rw-rr- 1 mapred mapred 7600 Nov 4 09:49 hadoop-hadoop-tasktracker-mroark-3-datanode3.log
      rw-rr- 1 mapred mapred 0 Nov 3 17:09 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-dfs-metrics.log
      rw-rr- 1 mapred mapred 4487748 Nov 4 09:49 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-jvm-metrics.log
      rw-rr- 1 mapred mapred 10486028 Nov 4 04:49 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-jvm-metrics.log.1
      rw-rr- 1 mapred mapred 4022120 Nov 4 09:49 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-mapred-metrics.log
      rw-rr- 1 mapred mapred 10486688 Nov 4 07:08 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-mapred-metrics.log.1
      rw-rr- 1 mapred mapred 8195840 Nov 4 09:49 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-rpc-metrics.log
      rw-rr- 1 mapred mapred 10486784 Nov 4 05:08 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-rpc-metrics.log.1
      rw-rr- 1 mapred root 49 Nov 4 09:49 hadoop-hadoop-tasktracker-mroark-3-datanode3.out
      rw-rr- 1 mapred root 49 Nov 3 17:09 hadoop-hadoop-tasktracker-mroark-3-datanode3.out.1

      after "sudo yum install hadoop-0.20"

      rw-rr- 1 hdfs hadoop 7600 Nov 4 09:49 hadoop-hadoop-tasktracker-mroark-3-datanode3.log
      rw-rr- 1 hdfs hadoop 0 Nov 3 17:09 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-dfs-metrics.log
      rw-rr- 1 hdfs hadoop 4487748 Nov 4 09:49 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-jvm-metrics.log
      rw-rr- 1 hdfs hadoop 10486028 Nov 4 04:49 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-jvm-metrics.log.1
      rw-rr- 1 hdfs hadoop 4022120 Nov 4 09:49 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-mapred-metrics.log
      rw-rr- 1 hdfs hadoop 10486688 Nov 4 07:08 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-mapred-metrics.log.1
      rw-rr- 1 hdfs hadoop 8195840 Nov 4 09:49 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-rpc-metrics.log
      rw-rr- 1 hdfs hadoop 10486784 Nov 4 05:08 hadoop-hadoop-tasktracker-mroark-3-datanode3.log-rpc-metrics.log.1
      rw-rr- 1 hdfs hadoop 49 Nov 4 09:49 hadoop-hadoop-tasktracker-mroark-3-datanode3.out
      rw-rr- 1 hdfs hadoop 49 Nov 3 17:09 hadoop-hadoop-tasktracker-mroark-3-datanode3.out.1

      This causes tasktracker to complain at startup, and it will no longer log anything, except in .out (for some reason).

      [mroark@mroark-3-datanode1 ~]$ sudo /etc/init.d/hadoop-0.20-tasktracker start
      Starting Hadoop tasktracker daemon (hadoop-tasktracker): starting tasktracker, logging to /usr/lib/hadoop-0.20/logs/hadoop-hadoop-tasktracker-mroark-3-datanode1.out
      log4j:ERROR setFile(null,true) call failed.
      java.io.FileNotFoundException: /usr/lib/hadoop-0.20/logs/hadoop-hadoop-tasktracker-mroark-3-datanode1.log (Permission denied)
      at java.io.FileOutputStream.openAppend(Native Method)
      at java.io.FileOutputStream.<init>(FileOutputStream.java:192)
      at java.io.FileOutputStream.<init>(FileOutputStream.java:116)
      at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
      at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:194)
      at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
      at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
      at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)

      This seems like bad behavior for an upgrade. The doc suggests to check file permissions after install, but I suggest the rpm should be smarter about what to change.

        Attachments

          Activity

            People

            • Assignee:
              plinnell Peter Linnell
              Reporter:
              mroark Mike Roark
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: