First, close: A couple of the existing implementations, but not all, implement both Closeable and DatasetReader. Because the close method is compatible with the Closeable interface, we should just have the DatasetReader implement Closeable.
Second, I think we should have DatasetReader<E> extend both Iterable<E> and Iterator<E>. This will cut down on errors because callers will be able to use for-each syntax:
The other nice thing is that other JVM languages use these interfaces to provide nicer APIs by default. So in ruby, you get an each implementation and all of the Enumerable methods that go with it (collect, map, etc.).
The main drawback is that all implementations would need to rename DatasetReader#read to #next (not too bad) and implement two extra methods: Iterable#iterator and Iterator#remove I usually handle these annoyances by adding an inner Base class to the interface, which I'll add in this case unless someone has a better option or complains.