Author Topic: v22 memory leak?  (Read 802 times)

IkerAriz

  • Community Member
  • Posts: 94
  • Hero Points: 0
v22 memory leak?
« on: November 06, 2017, 04:28:51 pm »
Just upgraded to v22 and memory consumption seems to have gone up from v21. Details:
  • Project uses wildcards and consists of ~4200 files.
  • Started editing session with 10 open files (python).
  • Approx 12k lines in all 10 files. Largest file is about 2k lines.
  • Many localized edits to one of those files over the course of 3 hours
  • Minor interaction with the "Projects" tool window (opening/closing folders, clicking on files to edit).
  • Frequent calls to build (Ctrl-M). I'm using a custom build script with just a few minor tweaks to error parsing rules.
I used "top" to keep an eye on se's RES during those three hours and watched it steadily grow from 190m to >700m. I stopped watching closely after that, but growth continues to well over 1GB over the course of the day (I just restart when I notice it that high).

Is there any way to help pin down where SE is using the memory?

Regards,
Iker

guth

  • Community Member
  • Posts: 44
  • Hero Points: 2
Re: v22 memory leak?
« Reply #1 on: November 06, 2017, 10:26:33 pm »
I have a similar experience. I had vs open, under Windows, for, maybe, two days. A Python project but mainly editing an HTML file with javascript. Did a lot of moving focus to the web browser and back. Noticed that vs grew a couple Mb every time alt-tabbing back. Attached image shows process sizes of two vs processes. One reached over 4 Gb with a handful of smallish source code files.

JeffB

  • Senior Community Member
  • Posts: 267
  • Hero Points: 9
Re: v22 memory leak?
« Reply #2 on: November 07, 2017, 06:46:13 pm »
I'm pretty sure I see a leak as well (Linux).  I had one instance at >5GB today.  Killed it and it came back with half that.  I had one up for several days last week and it eventually went >10GB.  Restarted it and it came back with 1-2GB (can't remember).

rowbearto

  • Senior Community Member
  • Posts: 804
  • Hero Points: 26
Re: v22 memory leak?
« Reply #3 on: November 08, 2017, 03:41:28 pm »
I think I'm seeing this too in Linux. A fresh SE restart and reloading my workspace uses 475K (with no editor windows open). But this morning after running over a day I was at 2.5GByte (but did have many editor windows open).

I'm doing a very long build now that fills up the process buffer, and memory is constantly growing, probably expected due to memory for the process buffer? Perhaps when I was at 2.5GByte my process buffer had too much?

patrick

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 817
  • Hero Points: 46
Re: v22 memory leak?
« Reply #4 on: November 08, 2017, 04:50:53 pm »
We're looking at some valgrind output, and we have found one leak.  Still checking to see if that's everything, but it's not a speedy process. It's like running the editor in molasses when I've got everything turned on for full leak checking.  Will update with more information once I have it.

Clark

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 4031
  • Hero Points: 267
Re: v22 memory leak?
« Reply #5 on: November 15, 2017, 04:20:30 pm »
One of these will fix this:

For 64-bit windows version do this:

* Backup your vsapi.dll
* Replace it with this:
    http://support.slickedit.com/Outbound/2200/vsapi_22000009.dll

For 64-bit Linux version do this:

* Backup your vs_exe
* Replace it with this:
    http://support.slickedit.com/Outbound/2200/vs_exe_22000009_linux64.bin

For Mac version do this:

* Backup SlickEditPro2017 installation. A simple rename will do this trick.
* Download and install this:
    http://support.slickedit.com/Outbound/2200/se_22000009_mac_2017_11_15.dmg

IkerAriz

  • Community Member
  • Posts: 94
  • Hero Points: 0
Re: v22 memory leak?
« Reply #6 on: November 16, 2017, 08:05:49 pm »
Thanks Clark - that definitely helped! However, it appears a smaller leak may remain (at least under ubuntu 16). Using the same setup as my earlier report "RES" memory grew from an initial 190 MB to 370MB over the course of ~6 hours. I'll keep an eye on it and report if it grows further.

Iker

patrick

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 817
  • Hero Points: 46
Re: v22 memory leak?
« Reply #7 on: November 16, 2017, 08:24:27 pm »
All other things being equal, it will grow a bit at first as the context tagging and other caches get populated through use, and then should level off.  If there's a remaining leak, it would be something that we didn't exercise when running instrumented, and we'll probably have to get some details from you about what languages are in your project, etc...   Thanks for the update.

IkerAriz

  • Community Member
  • Posts: 94
  • Hero Points: 0
Re: v22 memory leak?
« Reply #8 on: November 16, 2017, 08:57:33 pm »
Here's some info about my project:
  • Project has ~4300 files. Around 70% are C and C++, 10% python, the rest a mixture of shell, config and cmake files
  • All project files were added using wildcards. 26 filters altogether. For example "../trunk/vp/*" (SE project files are kept apart from source tree)
  • Using a custom build script which I invoke very frequently through build menu (Ctrl-M)
  • Small tweaks to error parser.
I've attached my current tagging options as well.

Regards,
Iker

IkerAriz

  • Community Member
  • Posts: 94
  • Hero Points: 0
Re: v22 memory leak?
« Reply #9 on: November 16, 2017, 09:00:22 pm »
Follow up.

In the time between my reply to Clark and now memory consumption has grown to 522 MB.

Iker

Dennis

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 2252
  • Hero Points: 283
Re: v22 memory leak?
« Reply #10 on: November 16, 2017, 09:51:42 pm »
We do a lot of caching, so it is normal for memory to grow to an extent.  If memory usage exceeds 2G in a day with a project your size, then perhaps there still is a lingering problem.

JeffB

  • Senior Community Member
  • Posts: 267
  • Hero Points: 9
Re: v22 memory leak?
« Reply #11 on: November 17, 2017, 06:22:55 pm »
This seems to work for me as best I can tell.  I haven't seen an instance go >3GB yet.

jporkkahtc

  • Senior Community Member
  • Posts: 1425
  • Hero Points: 112
  • Text
Re: v22 memory leak?
« Reply #12 on: November 17, 2017, 07:55:05 pm »
WRT Caching: I did a FindFile to look for *.X files in a collection of ZIP archives.
Then from Explorer I tried deleting some of these files, but delete failed because Slick was holding them open (even though it had no buffers open for them).

Using PROCEXP I could see Slick holding handles to these files, and releasing them one at a time.
So I happened to see some Slick caching in action - it seems to hold *.ZIP files open for some short time, then closes them.

JeffB

  • Senior Community Member
  • Posts: 267
  • Hero Points: 9
Re: v22 memory leak?
« Reply #13 on: November 17, 2017, 08:49:14 pm »
Spoke to soon.  The instance at 3GB a little while ago is now close to 5GB.  I can load files and do references and every now and then it'll jump a few hundred MB, but I don't see a pattern.

This is a large project ~200k files.  Tag file is ~2.5GB (which is also the Tag file cache maximum)

Dennis

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 2252
  • Hero Points: 283
Re: v22 memory leak?
« Reply #14 on: November 17, 2017, 08:54:45 pm »
Does that 5GB include memory allocated to memory mapped files?   Tag files are memory mapped, unless you have them configured otherwise.  With a tag file cache of 2.5G, it is conceivable that you got to 5G rather organically just by filling up the tag file cache, in addition to memory allocated to other memory mapped tag files and other caching and normal memory usage.