Author Topic: Displaying the character "x0D" as ":"  (Read 6316 times)

rgloden

  • Senior Community Member
  • Posts: 169
  • Hero Points: 5
Displaying the character "x0D" as ":"
« on: April 02, 2016, 09:22:32 PM »
SE 20.0.1.3 64bit, Hotfix 7, Windows 7

I have a couple of Perl source files (extention ".pm") which recently appear to have scattered through them the character "x0D" which is not at the end of a line, e.g. it's not the normal x0Dx0A sequence.  SE is displaying this as a ":" character which register as a problem until I tried to out the strings today to a console.  I'm guessing this happened as a result of a search/replace of the "\n" character several days ago.  (Once I understood the problem today, I did a find/replace of the \x0D with "" and the file seems to be fine again.)

This correlates with other search/replace operations recently which seem to insert the ":" in place of the "\n" ... though I caught those instances and reverted immediately.

Questions:
1) Is this some Unicode thing biting me?
2) If so, how do I tell SE this is not a Unicode formatted file?
3) How can I tell how SE is interpreting an input file? e.g. DOS, UNIX, Unicode, etc.
4) How can I instruct SE that the file is just plain DOS formatted ASCII.
« Last Edit: April 02, 2016, 10:22:36 PM by rgloden »

Clark

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 7089
  • Hero Points: 537
Re: Displaying the character "x0D" as ":"
« Reply #1 on: April 02, 2016, 10:02:36 PM »
By default, SlickEdit does display erroneous carriage returns as you described for non Unicode files. If you think SlickEdit is adding this on you, please provide steps to reproduce.

1) sounds like you're editing non Unicode files on Windows
2) display the save as dialog for the current file. It will display "Text, SBCS/DBCS" which indicates it's not a Unicode file
3)  Hit end key. If hex byte displayed in status is 0D, cursor right and next byte will likely be 0A. That tells you the file is being recognized as a <CR><LF> file.
4) open the file using the windows open dialog and specify the line format to force a line format. This is rarely necessary except when a file has mixed line endings.
« Last Edit: April 02, 2016, 10:04:43 PM by Clark »

rgloden

  • Senior Community Member
  • Posts: 169
  • Hero Points: 5
Re: Displaying the character "x0D" as ":"
« Reply #2 on: April 02, 2016, 11:06:00 PM »
Hmmm ... in my environment it seems that replacing "\n" with "%%%" always  leaves the \x0D (independent of the steps below).

0) Running SE on Windows environment with mixed DOS and Linux files.
1) I copied several lines of C++ comments from a Linux formatted file and pasted into DOS perl file.  The destination file still indicates "Text, SBCS/DBCS mode"
2) Select the lines just pasted and Search/Replace "\n" with "%%%" with Search attributes: Perl RegExp, Highlight replaced text, List replaced Matches.
3) The Search Results windows indicates the replaced text is now "<CR>%%%" (the "<CR>" sometimes is only displayed the first occurrence?)"
4) The resulting text now displays as ":%%%" and viewing in line hex mode the ":" is the \x0D character.

So I tried the same using SE on Linux.
1) Create a myTest.txt file in NotePad on DOS and insert several simple lines, e.g.
This is a test
This is a test
This is a test
This is a test
This is a test
2) Open the DOS file using SE on Linux (20.0.1.3 with Hotfix 7)
3) Select "Save As" to check file type and see that format is "UTF-8 no signature" (this file shows "Text, SBCS/DBCS" in Windows SE).  Don't actually save.
5) View in Line Hex mode to verify line endings is \x0D\x0A ... and it is.
6) Select "Save As" (while in Line Hex view) and SE indicates "Text, SBCS/DBCS".  But don't save.
7) Change back to regular view
8) Select the 3 middle lines and do the replace "\n" with "%%%" with Perl RegEx selected
9) The Search Results window shows the "<CR>%%%" for every replacement.
10) In the actual file view, the replaced data is also "<CR>%%%"
11) Linux SE makes us save the file before views in Line Hex.  Viewing in Line Hex mode doesn't show the "<CR>" but instead the ":" which is the "\x0D" character.
12) Alternate: If you replace without doing a selection, the results vary slightly and after the save which Line Hex make you do, the line changes from a single line to multiple lines with \x0D now acting as a CR/LF, e.g.
This is a test
%%%This is a test
%%% This is a test
%%% This is a test ...
« Last Edit: April 03, 2016, 01:21:25 AM by rgloden »

Clark

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 7089
  • Hero Points: 537
Re: Displaying the character "x0D" as ":"
« Reply #3 on: April 03, 2016, 01:34:49 AM »
You meant to use \R. Regex syntax changed slightly.

Clark

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 7089
  • Hero Points: 537
Re: Displaying the character "x0D" as ":"
« Reply #4 on: April 03, 2016, 02:33:12 AM »
Line Hex mode reloads the file as Text, SBCS/DBCS. If the file was modified, you'd notice that switching to Line Hex mode prompts you to save because it needs to reload the file in Binary Text, SBCS/DBCS. Displaying Utf-8 characters in Line Hex mode wouldn't be practical. It would need up to 12 hex lines below each line since a Utf-8 character can be up-to 6 bytes long (and composite characters are an additional problem).

rgloden

  • Senior Community Member
  • Posts: 169
  • Hero Points: 5
Re: Displaying the character "x0D" as ":"
« Reply #5 on: April 05, 2016, 03:18:32 AM »
Thanks for the explanation ... though I'm a little sad that I have to retrain my "old" brain.  I'm sure this will trip me up a few times.

Note: The "Release Notes" seems to indicate that "\R" is the "same as \n" though the online help has a good explanation of the changes and delta's from Perl.

Clark

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 7089
  • Hero Points: 537
Re: Displaying the character "x0D" as ":"
« Reply #6 on: April 05, 2016, 01:46:19 PM »
The real mistake was years ago. Sorry we had to fix it to adhere to the perl standard.