Details
-
Type:
Improvement
-
Status: Resolved
-
Priority:
Trivial
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 1.2.0
-
Component/s: Data Module
-
Labels:None
Description
As a safeguard for data removal it'd be nice if Kite moved deleted partitions to trash[1] instead of directly dropping them from HDFS completely. This would provide a safeguard in case a delete command was issued incorrectly and completed or some data was deleted in error based on incorrect configuration.
This could be implemented in a couple of ways:
- Dataset.deleteAll would move things to trash instead of calling FileSystem.delete(...)
- A new Dataset.moveToTrash() method gets added (or Dataset.deleteAll(boolean skipTrash)) that could be called for people wanting to be safer with deletes.
[1] - https://hadoop.apache.org/docs/r2.4.1/api/org/apache/hadoop/fs/Trash.html
Attachments
Issue Links
- relates to
-
KITE-1103 Moving an entire Kite dataset to trash excludes .metadata directory
-
- Resolved
-
- links to