Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 0.14.1
-
Fix Version/s: 0.15.0
-
Component/s: Data Module
-
Labels:None
Description
As I was playing around with some changes related to CDK-450, I noticed that FilesystemDataset.deleteAll() results in IllegalStateException:
java.lang.IllegalStateException: Attempt to retrieve the partition strategy on a non-partitioned descriptor:DatasetDescriptor{format=Format
{name=avro}, schema={"type":"record","name":"user","fields":[
{"name":"username","type":"string"},
{"name":"email","type":"string"}]}, location=hdfs://localhost:51583/var/folders/9k/wlyxzx3d4mb2kx3jkdk9t9t56bkgvq/T/1402351372217-0, properties={}, partitionStrategy=null}
at com.google.common.base.Preconditions.checkState(Preconditions.java:172)
at org.kitesdk.data.DatasetDescriptor.getPartitionStrategy(DatasetDescriptor.java:210)
at org.kitesdk.data.spi.filesystem.FileSystemView.partitionIterator(FileSystemView.java:148)
at org.kitesdk.data.spi.filesystem.FileSystemView.deleteAllUnsafe(FileSystemView.java:156)
at org.kitesdk.data.spi.filesystem.FileSystemDataset.deleteAll(FileSystemDataset.java:130)
at org.kitesdk.data.spi.filesystem.TestFileSystemDataset.testDeleteAllWithoutPartitions(TestFileSystemDataset.java:506)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:24)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
The JavaDoc says that deleteAll() may throw UnsupportedOperationException but IllegalStateException would never be expected. I assume the intention here would be that deleteAll() would be supported for this case and it would result in all of the data files in the Dataset being deleted.
I have not gone back to look if this ever worked to know how it might have behaved before. It is possible that it may be desired to treat this as unsupported since it would not be an atomic operation?