Author Topic: Special Characters ... and display bugs with UTF16  (Read 859 times)

jporkkahtc

  • Senior Community Member
  • Posts: 1425
  • Hero Points: 112
  • Text
Special Characters ... and display bugs with UTF16
« on: December 06, 2013, 12:49:15 am »
I'm using slick 18.0.1.2 on Windows.
I've got a UTF-16 file (tab delimited CSV), 11MB
The buffer is loaded in "Plain text" mode.
The special character for Unicode tab is "<HT>"

The lines are pretty long, and I was trying to figure out why some text isn't importing into Excel correctly.
Each line has about 35 columns (so, 35 tab separated fields).

After character column 256, the tab character displays as "<HT>".
To be clear... a line of text is display like this: (Where "~" is the line slick displays for tabs for plain text, 8bit, files).
AAA~BBB~ ...... ~ZZZ<HT>111<HT>
The "ZZZ" text appears on col 256 (as reported in the status line by Slick).


Why don't tabs display correctly until column 256?


Hm....
worse... it seems Slick doesn't display text correctly on long lines in this case. Enabling display-special-characters fixes that problem.
Repro: Create UTF-16 text file, create long lines with lots of tabs.
Starting around column 245 shift-right arrow to select and move the cursor, and watch the text on the line start shifting around as you go past col 256 and encounter tabs.