SlickEdit Community

SlickEdit Product Discussion => SlickEdit® => Topic started by: rjpontefract on February 12, 2021, 02:40:45 am

Title: Slick-C stack when saving large file
Post by: rjpontefract on February 12, 2021, 02:40:45 am
I was editing a rather large file (over 110 million lines, for which I have respect for SE) when I hit F2 instead of F3.  F2 seems to save the file which SE did, and then presented the following stack.  I couldn't do anything with the stack dialog and had to kill SE and restart it.

Code: [Select]
stack[2021-02-12T12:52:02Z] Slick-C STACK TRACE ******************************
stack[2021-02-12T12:52:02Z] Created on 2/12/2021 at 12:52:02 (2 ms)
stack[2021-02-12T12:52:02Z] SlickEdit Pro Version 25.0.1.0 Copyright 1988-2020 SlickEdit Inc.
stack[2021-02-12T12:52:02Z] Edit module and type "st -f <offset>" to get the
stack[2021-02-12T12:52:02Z] run-time error position
stack[2021-02-12T12:52:02Z]
stack[2021-02-12T12:52:02Z] error code=-3009
stack[2021-02-12T12:52:02Z] Invalid number argument
stack[2021-02-12T12:52:02Z]
stack[2021-02-12T12:52:02Z]vsvcs.dll 0x000000006671AD90 DSCreateDelta(C:\Users\me\Downloads\log.txt)   p_window_id: 21   p_object: OI_EDITOR   p_name:
stack[2021-02-12T12:52:02Z]saveload.ex 49 save_file(C:\Users\me\Downloads\log.txt,-O +DD -Z -ZR -E -S +fenddefaults )   p_window_id: 21   p_object: OI_EDITOR   p_name:
stack[2021-02-12T12:52:02Z]files.ex 8581 save(,8)   p_window_id: 305   p_object: OI_EDITOR   p_name: _search0
stack[2021-02-12T12:52:02Z]window.ex 1827 _window_quit(0,1,0,0)   p_window_id: 21   p_object: OI_EDITOR   p_name:
stack[2021-02-12T12:52:02Z]moveedge.ex 653 close_window(,1)   p_window_id: 21   p_object: OI_EDITOR   p_name:
stack[2021-02-12T12:52:02Z]window.ex 9692 _on_close()   p_window_id: 21   p_object: OI_EDITOR   p_name:
Title: Re: Slick-C stack when saving large file
Post by: Clark on February 12, 2021, 01:34:29 pm
Looks like it's having a problem making a backup.

What version of SlickEdit are you using?

What is the file size?

What is your "Maximum size to backup (KB)" set to?  Tools>Options>File Options>Backup

Can you reproduce this problem everytime? If so, does this happen when you save other small files?

Title: Re: Slick-C stack when saving large file
Post by: rjpontefract on February 14, 2021, 09:04:57 pm
SE is version 25.0.1.0 with hot fix 1.
The file is just over 19GiB.
Maximum backup size is 20000KiB.  Although Limit size of Backup was off, so I guess it's not used.
Yes it happens every time with this file.  I have never seen an issue with any other files.
I'll turn Limit size of Backup on so that it doesn't try backing up this monster of a file, although I wouldn't expect SE to die when it tries.
Title: Re: Slick-C stack when saving large file
Post by: Clark on February 14, 2021, 11:31:42 pm
The backup system has optimizations in it for supporting very large files. We test with files much larger than yours.

It sounds like this particular files backup data is bad. We will need to investigate to try and determine where things are going wrong.


 
Title: Re: Slick-C stack when saving large file
Post by: rjpontefract on February 14, 2021, 11:37:05 pm
Good to hear that you support very large files, I don't come across them very often, but when I do it's good that SE handles them well.
What information can I provide that would help you to determine what's going wrong in this case?
Title: Re: Slick-C stack when saving large file
Post by: Dan on February 15, 2021, 01:22:44 pm
It's good that you have a reproducible case.  That should make things easier.  Any chance you can send me the file or is does it have proprietary information in it?
Title: Re: Slick-C stack when saving large file
Post by: rjpontefract on February 15, 2021, 09:00:23 pm
Hi Dan, unfortunately the file contains proprietary information.  I'll see if I can create another example log file that I can share.
Title: Re: Slick-C stack when saving large file
Post by: Dan on February 15, 2021, 09:04:26 pm
OK.  Please let me know.
Title: Re: Slick-C stack when saving large file
Post by: rjpontefract on February 15, 2021, 10:43:37 pm
I created a test file with 340,000,000 lines containing:

"This is an example log line to help reproduce an SE case"

I added " and an edit" to the end of a random line near the bottom and hit save.  A while later I got the stack:

Code: [Select]
stack[2021-02-16T11:40:44Z] Slick-C STACK TRACE ******************************
stack[2021-02-16T11:40:44Z] Created on 2/16/2021 at 11:40:43 (981 ms)
stack[2021-02-16T11:40:44Z] SlickEdit Pro Version 25.0.1.0 Copyright 1988-2020 SlickEdit Inc.
stack[2021-02-16T11:40:44Z] Edit module and type "st -f <offset>" to get the
stack[2021-02-16T11:40:44Z] run-time error position
stack[2021-02-16T11:40:44Z]
stack[2021-02-16T11:40:44Z] error code=-3009
stack[2021-02-16T11:40:44Z] Invalid number argument
stack[2021-02-16T11:40:44Z]
stack[2021-02-16T11:40:44Z]vsvcs.dll 0x0000000063D8AD90 DSCreateDelta(C:\Temporary\test2.log)   p_window_id: 735   p_object: OI_EDITOR   p_name:
stack[2021-02-16T11:40:44Z]saveload.ex 49 save_file(C:\Temporary\test2.log,-O +DD -Z -ZR -E -S +fenddefaults )   p_window_id: 735   p_object: OI_EDITOR   p_name:
stack[2021-02-16T11:40:44Z]files.ex 8581 save(,8)   p_window_id: 735   p_object: OI_EDITOR   p_name:

Hope that helps.
Title: Re: Slick-C stack when saving large file
Post by: Dan on February 15, 2021, 10:48:21 pm
I'm working on trying to reproduce this now.
Title: Re: Slick-C stack when saving large file
Post by: Dan on February 15, 2021, 11:16:48 pm
Can you send me your user.cfg.xml file?
Title: Re: Slick-C stack when saving large file
Post by: rjpontefract on February 15, 2021, 11:25:38 pm
Hi Dan

Is there an email address I can send it to please?  It seems that PMs in the forum don't support attachments.
Title: Re: Slick-C stack when saving large file
Post by: Clark on February 16, 2021, 01:07:08 am
SE is version 25.0.1.0 with hot fix 1.
The file is just over 19GiB.
Maximum backup size is 20000KiB.  Although Limit size of Backup was off, so I guess it's not used.
Yes it happens every time with this file.  I have never seen an issue with any other files.
I'll turn Limit size of Backup on so that it doesn't try backing up this monster of a file, although I wouldn't expect SE to die when it tries.

Somehow I misread the file size ;D I thought I saw megabytes. Now I see your file is 19 gigabytes and not 19 megabytes. Once I create a 19 gigabyte test file, I can reproduce the problem.

Thanks for the steps for reproducing the problem.
Title: Re: Slick-C stack when saving large file
Post by: rjpontefract on February 16, 2021, 01:10:27 am
Hi Clark
No worries, it's easily done.  I did wonder when you said you test with files much larger than mine :-)
Title: Re: Slick-C stack when saving large file
Post by: Clark on February 16, 2021, 03:29:09 pm
Right now, the backup system is limited to 2 gigabyte files (signed integer).

We will try to improve this for the next release (v26). I'm pretty sure this can be done for 64-bit versions of SlickEdit. "File>Backup History For ..." uses DiffZilla. I'm not expecting that to work reasonably for files this large. DiffZilla requires a lot of memory and can take a long time to diff files this large.
Title: Re: Slick-C stack when saving large file
Post by: rjpontefract on February 16, 2021, 07:36:45 pm
I've enabled the "Limit size of backup file" option to avoid this issue.  Files this large are always log files and I'm not usually interested in backing them up.  I do add comments to them though, so being able to save them without causing SE to hangup is the important part.