SlickEdit Community
Archived Beta Discussions => SlickEdit 201x Beta Discussions => SlickEdit 2017 v22 Beta Discussion => Topic started by: jporkkahtc on September 20, 2017, 06:19:19 PM
-
I created a plain text file,
size=218,264,095 bytes.
#lines=7,385,617
Saving in Beta3 takes 40 seconds!
Saving the same file in V21: Less than 1 second.
Happens even if I turn off backup history.
Happens with UTF-8 and DBCS encoding. (The data is plain ascii text).
Happens every time I save the file, or SaveAs to a different name.
From an external program I can see the size of the file being saved slowly growing from 0 to 218mb.
I don't see any significant difference in options between v21 and v22 - same FileOtions and ApplicationOptions.
-
Looking with PROCMON, saving the file (with dbcs encoding).
Slick writes about 32kb at a time ... but not 32,768 bytes -- seemingly random lengths just under 32,768 bytes each time.
It oughta write 32,768 bytes each time, and keep it aligned.
Also, at the start of the save it appears to truncate the file to 0 bytes - OK, but before writing that much it would help to set the file length to the desired size for best disk space allocation.
V21 writes the same file 8,192 bytes at a time - aligned with DBCS, but offset with utf8 by 3 bytes - but either way it is fast.
This is writing to an SSD.
-
Just to be REALLY sure, while the file is open run "save +o" on the SlickEdit command line.
-
Same result.
-
I've read through this quickly. Have you tried a default config?
I just tried saving foo3 and on my machine with a clean config, the save is a fraction of a second.
-
Note: SlickEdit was on plain text mode for foo3
-
OK, clean configuration doesn't repro --- in this case Slick is doing 8KB IOs.
I tried importing all options except appearance, keyboard, debugging, languages.
It still did not repro.
I then imported all options, and now it does repro.
-
My best guess is a save option
-
Which?
Short of importing all settings, it doesn't repro.
I tried:
Repro: Importing all
Not repro: Importing all except appearance, keyboard, debugging, languages
Not repro: Importing only appearance, keyboard, debugging, languages
-
Are you changing the line endings on save? If so, it will have to write a spill file which takes a long time. You can increase the buffer cache size (default is 2 megabytes) to speed this up.
-
Nope.
PROCMON shows the only IO is Slick writing just slightly less than 32KB at a time.
From a clean config it actually alternates reading the original file and writing at 8kb at a time -- much faster somehow.
In my config
Buffer cache size (MB)"=8000 (default=2)
Load partially when files are larger than=10,001 (default is 8,000)
Wrap line length=8,000 (default is 4,000)
Maximum size to backup=75,000(default 20,000)
THese are the only difference from default config which looked relevant to me, yet setting these isn't enough to make it repro.
-
I uploaded my options export to support/porkka_slow_save.
-
I haven't looked at the options you exported, but strip trailing spaces would be a killer on a large file too.
-
nope, i don't have that set.
-
You are translating the line endings to Unix EOL on save (see language specific save options). The save code could be optimized but hasn't been yet (i.e only change the data when the eol characters actually change). I don't know if this is easy, hard, or too hard to be worth doing.
-
Ah.... So this changed when I was trying to figure out encoding and line endings (see other posts about this).
Confirmed: Changed to "automatic" and the problem goes away.
SaveAs->LF (even though the file is already LF) is also slow - presumably this is the same code path.
I wanted to get so all newly created files are utf-8 and LF endings.
So, what does "AUTOMATIC" do here?
The Help text in Options shows:
Specifies how end of line characters are translated when a file is saved. Automatically attempts to determine the context.
I have no idea what that means ... what context?
Windows vs Linux?
-
For existing files,EOL characters are determined from first line of file. Otherwise, use OS default line endings. Also, save won't translate lines with mixed line endings unless a specific line ending format is specified. When all save translations are off. Save is fastest. When there are save translations what modify the original file, a lot of data that must be spilled may be generated in addition to the file contents being written to disk.
-
Looks like there is a bug in the optimization code. The save code was trying to optimize this particular case but it was goofed so it thinks every line is being modified when saving an LF file where the original file already just has LF line endings. The CR LF case is optimized correctly.
-
This optimization has been fixed in the next build