Errors in reading CSV files, possibly because of encoding
Hello,
I requested help with my initial problem in the ParaView support forum when I could not convert a successfully loaded CSV files to points:
https://discourse.paraview.org/t/cannot-convert-csv-table-to-points/3803
On a whim, I decided to enclose the column headers in the file with double quotes, even though that is not required under the CSV standard. The loaded data then successfully converted to points. The only difference I could see in the table of loaded data was that, in the first case, the columns and headers were in what seemed random order. Once adding the quotes, the columns were sorted in alphabetical order. After looking into the code, the problem seemed to be happening in vtkIOInfoVis::DelimitedTextIterator.InsertField(). After further research, I came across this:
https://discourse.paraview.org/t/how-can-i-read-non-ascii-filenames-from-pvpython/3121
I am using Windows 10. I looked at my CSV files. The small 4x4 files I was using to test this were made in Notepad. They are saved in UTF-8 by default and that format encountered the issues described above. However, when I switched it to using Unicode, I no longer needed to enclose the column headers in quotes for the table-to-point filter to be successful. Therefore, it seems that there is a bug in correctly reading character encoding, at least in Windows.
Thanks.