Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 0.13.0
-
Fix Version/s: 0.14.0
-
Component/s: None
-
Labels:None
Description
CSV schema inference always produces nullable fields, which are not compatible with the non-null equivalents. In the CSV import process, we read with the schema of the Dataset it will be stored in, but first check that the inferred schema is compatible. To avoid assumed-nullable fields causing incompatible schema errors, we should instead check a few lines to see if there are any missing values.