There seems to be an issue with Hadoop append and using hsync. The file data is updated but the ls file stats are not being updated. Attached is a small java program to illustrate the issue. Also attached is the input and output of running the program.
The expected output is the file size and date updated should be updated after a hsync when a ls is done.
Note that this in turn causes the map part of a mapreduce jobs not to read any appended data because the file length does not change.