SlickEdit Community
SlickEdit Product Discussion => SlickEdit® => Topic started by: IkerAriz on November 06, 2017, 04:28:51 PM
-
Just upgraded to v22 and memory consumption seems to have gone up from v21. Details:
- Project uses wildcards and consists of ~4200 files.
- Started editing session with 10 open files (python).
- Approx 12k lines in all 10 files. Largest file is about 2k lines.
- Many localized edits to one of those files over the course of 3 hours
- Minor interaction with the "Projects" tool window (opening/closing folders, clicking on files to edit).
- Frequent calls to build (Ctrl-M). I'm using a custom build script with just a few minor tweaks to error parsing rules.
I used "top" to keep an eye on se's RES during those three hours and watched it steadily grow from 190m to >700m. I stopped watching closely after that, but growth continues to well over 1GB over the course of the day (I just restart when I notice it that high).
Is there any way to help pin down where SE is using the memory?
Regards,
Iker
-
I have a similar experience. I had vs open, under Windows, for, maybe, two days. A Python project but mainly editing an HTML file with javascript. Did a lot of moving focus to the web browser and back. Noticed that vs grew a couple Mb every time alt-tabbing back. Attached image shows process sizes of two vs processes. One reached over 4 Gb with a handful of smallish source code files.
-
I'm pretty sure I see a leak as well (Linux). I had one instance at >5GB today. Killed it and it came back with half that. I had one up for several days last week and it eventually went >10GB. Restarted it and it came back with 1-2GB (can't remember).
-
I think I'm seeing this too in Linux. A fresh SE restart and reloading my workspace uses 475K (with no editor windows open). But this morning after running over a day I was at 2.5GByte (but did have many editor windows open).
I'm doing a very long build now that fills up the process buffer, and memory is constantly growing, probably expected due to memory for the process buffer? Perhaps when I was at 2.5GByte my process buffer had too much?
-
We're looking at some valgrind output, and we have found one leak. Still checking to see if that's everything, but it's not a speedy process. It's like running the editor in molasses when I've got everything turned on for full leak checking. Will update with more information once I have it.
-
One of these will fix this:
For 64-bit windows version do this:
* Backup your vsapi.dll
* Replace it with this:
http://support.slickedit.com/Outbound/2200/vsapi_22000009.dll (http://support.slickedit.com/Outbound/2200/vsapi_22000009.dll)
For 64-bit Linux version do this:
* Backup your vs_exe
* Replace it with this:
http://support.slickedit.com/Outbound/2200/vs_exe_22000009_linux64.bin (http://support.slickedit.com/Outbound/2200/vs_exe_22000009_linux64.bin)
For Mac version do this:
* Backup SlickEditPro2017 installation. A simple rename will do this trick.
* Download and install this:
http://support.slickedit.com/Outbound/2200/se_22000009_mac_2017_11_15.dmg (http://support.slickedit.com/Outbound/2200/se_22000009_mac_2017_11_15.dmg)
-
Thanks Clark - that definitely helped! However, it appears a smaller leak may remain (at least under ubuntu 16). Using the same setup as my earlier report "RES" memory grew from an initial 190 MB to 370MB over the course of ~6 hours. I'll keep an eye on it and report if it grows further.
Iker
-
All other things being equal, it will grow a bit at first as the context tagging and other caches get populated through use, and then should level off. If there's a remaining leak, it would be something that we didn't exercise when running instrumented, and we'll probably have to get some details from you about what languages are in your project, etc... Thanks for the update.
-
Here's some info about my project:
- Project has ~4300 files. Around 70% are C and C++, 10% python, the rest a mixture of shell, config and cmake files
- All project files were added using wildcards. 26 filters altogether. For example "../trunk/vp/*" (SE project files are kept apart from source tree)
- Using a custom build script which I invoke very frequently through build menu (Ctrl-M)
- Small tweaks to error parser.
I've attached my current tagging options as well.
Regards,
Iker
-
Follow up.
In the time between my reply to Clark and now memory consumption has grown to 522 MB.
Iker
-
We do a lot of caching, so it is normal for memory to grow to an extent. If memory usage exceeds 2G in a day with a project your size, then perhaps there still is a lingering problem.
-
This seems to work for me as best I can tell. I haven't seen an instance go >3GB yet.
-
WRT Caching: I did a FindFile to look for *.X files in a collection of ZIP archives.
Then from Explorer I tried deleting some of these files, but delete failed because Slick was holding them open (even though it had no buffers open for them).
Using PROCEXP I could see Slick holding handles to these files, and releasing them one at a time.
So I happened to see some Slick caching in action - it seems to hold *.ZIP files open for some short time, then closes them.
-
Spoke to soon. The instance at 3GB a little while ago is now close to 5GB. I can load files and do references and every now and then it'll jump a few hundred MB, but I don't see a pattern.
This is a large project ~200k files. Tag file is ~2.5GB (which is also the Tag file cache maximum)
-
Does that 5GB include memory allocated to memory mapped files? Tag files are memory mapped, unless you have them configured otherwise. With a tag file cache of 2.5G, it is conceivable that you got to 5G rather organically just by filling up the tag file cache, in addition to memory allocated to other memory mapped tag files and other caching and normal memory usage.
-
I assume it does not. FYI....The Virtual Size is just over 6GB.
I'll leave it up for a while and see if it keeps growing.
-
FYI...After sitting for a few days with only minor usage, the resident memory is 6.35GB and Virtual is 7.5GB.
-
Additional follow up...
Total RES memory for my VS process is now a bit over 1GB and continues to grow, albeit slowly. A quick peek at the "/proc/pid/smaps" file suggests most of that is heap (see output below).
Regards,
Iker
02f18000-3b924000 rwxp 00000000 00:00 0 [heap]
Size: 927792 kB
Rss: 927084 kB
Pss: 927084 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 927084 kB
Referenced: 920808 kB
Anonymous: 927084 kB
AnonHugePages: 864256 kB
Shared_Hugetlb: 0 kB
Private_Hugetlb: 0 kB
Swap: 0 kB
SwapPss: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
VmFlags: rd wr ex mr mw me ac sd
-
Thanks for the updates. We found the leak that probably accounts for your continued resident memory growth, and it's been fixed for the point release.
For our current tests, the leak was slow enough where it got lost in the noise of memory we keep in caches. But editing large source files, or files that are really tag dense (like XML files) can make the leak fast enough to observe just watching the OS reports on memory usage. Just running for a long time with normal usage will leak as well, just not as quickly.
-
Great to hear! Thanks for the follow up.
Regards,
Iker
-
Hi Patrick,
I switched to version 22.0.1 and while the memory leak seems to have slowed down it's still present (eg, 1GB+ heap after a few days). It also appears that as memory use grows editing becomes more janky/less responsive: cursor movement and scrolling stagger, noticeable delays in autocomplete, etc.
Are there any diagnostics I can enable to help track the issue?
Regards,
Iker
-
FYI...
The leak seems to be gone for me. I have noticed unusual delays and extreme lapses in response, but I'm not sure if that's related only to Python files or not.
Jeff
-
@IkerAriz, probably the best option is to turn on Slick-C profiling when it starts getting janky, do whatever operations are showing delays for you for a few minutes, and then turn it off and send us the profile. If there is a relation between the memory use and the jankiness, then that could point us in the right direction. You can access the profiler controls in the Macro -> Slick-C Profile submenu. In the meantime, I'll re-run some of the memory tests with a project setup like you described, to see if I can spot anything I've missed.
I see from one of your previous posts that background tagging is disabled, so there are a few things that can block the main thread with that. After the profiling, it narrow things further to see if that makes a difference.
-
Hi Patrick,
I enabled background tagging and UI performance didn't get better (AFAICT) but memory consumption did appear to grow faster. Attached is a slick-c profile (BG tagging enabled) that captures a few examples of significant latency.
Thanks,
Iker
-
Thanks, looking at the profile.
Quick question: do you have C# files in your project, or were you editing any?
-
My project has 4 C# files but I was not editing them (the files are part of the project but they're practically never touched). At the time I produced the profile I was editing a C++ file.
-
Looking over your profiling results also. My suggestion to eliminate the hiccups. Document > C/C++ Options > Auto-Complete > Auto-list compatible values > OFF (uncheck). Also turn off Document > C/C++ Options > Context Tagging > Auto-list compatible parameters.
While nice to have, these two features require a lot of calculation, especially with templates, which I can see from your profiling data that you have a lot of.
You can manually force SlickEdit to list compatible values by hitting Ctrl+,
-
Thanks Dennis. Disabling those two features improved responsiveness. However, the memory issue remains with heap consumption regularly exceeding 1GB.
BTW, I noticed something that may be of interest. I recently added another C++ project to my workspace with ~1200 files and made it inactive (ie, my first project with ~3800 files is the active one). I quit VSE and then went through the following steps:
- Start VSE with no open files.
- Check memory. Heap use is 175 MB.
- Click open a single 63-line C++ file in my first project. No edits/movement in file.
- Check memory. Heap use is 178 MB.
- Click open a single 125-line C++ file in newly added project. No movements/edits in file.
- Check memory. Heap use is 266 MB.
When I make the new project "active" and repeat the steps above the big memory jump happens in step 4.
Iker
-
After disabling "Auto-list compatible values", "Auto-list compatible parameters" and "C/C++->auto-complete" typing latency is still significant; in particular, when hitting an opening parentheses, a member access period or arrow.
Attached is another profile that captures latency with this configuration. Note that heap memory for this instance of VSE is nearly 1.4GB (after a couple of days).
-
Are you using Boost, or heavily using the STL in your code?
-
No boost. But the project does make liberal use of the more common STL types (string, vector, list, map, etc).
-
Any luck with the latest profile? Is there other info I can provide to help with the debugging?
Iker
-
I was not able to gather much from that profile. I can say that STL context tagging is tough. If you ever try to manually trace down what actually happens just to do something like: vector v [ x ] -> <list symbols>, you would have a long day of work before you untangled it all, if you were able to at all. It is just a maze of typedefs and allocator templates, nothing goes directly from point A to point B. And now we also have to do type inference with that. We have some shortcuts in there to make some things faster, but perhaps we need to do even more. We have another outstanding complaint that tagging is slow with Visual Studio 2017 headers. This happens occasionally, compiler vendors change their STL implementation, and it seems to always adds more complexity.
One thing you can do to make things easier, is to make sure that you do not have duplicate STL implementations tagged in your workspace. Set up and use your compiler configuration tag file, and do not tag another set of compiler headers in addition. It is hard enough for tagging to untangle this mess, and even harder if there are multiple STL implementations to disambiguate.
-
Here are my configuration changes thus far:
- Disabled the options as recommended below (auto compat values, auto compat params, etc)
- Removed all STL from the compiler tag files
- Trimmed the compiler tag file to include only C files under /usr/include; ie, top level files, python C api headers, the gcc C headers (the "sys" and "asm" dirs under "x86_64-linux-gnu") plus a few others.
- Removed two C++ projects from my workspace (eigen and kaldi); workspace now has only one project (my own)
Editing feels faster, if a bit more spartan. There's also noticeable delay when hitting "alt-," to manually trigger function argument help (though this is less of a problem in practice). Memory consumption, surprisingly, appears to have worsened. My reduced workspace and and trimmed tag files are both considerably smaller than they were at the start of this thread yet heap memory on startup (with no files open) is ~230 MB compared to the earlier 175 MB. Heap consumption after startup grows continuously as before.
Thanks,
Iker
-
Any news on this front?
Iker
-
@IkerAriz: What is your tag file cache set to? You can, I do not entirely recommend it, but you can set it as low as 8M for a maximum tag file cache size. You can also turn off the option to use memory mapped file I/O to further reduce the tag file caching footprint. The memory footprint numbers you are reporting are in my opinion pretty normal amounts. With most computers having at 8G plus of memory, we are trying to improve performance by caching in a lot of areas in the editor, so it is normal for the memory footprint to grow over time. However, the tag file maximum is one way that you can absolutely reduce the footprint, if you are willing to sacrifice a certain amount of performance.
-
Dennis,
Both "tag file cache size" and "tag file cache maximum" are set to 64 MB.
My concern is not that SE consumes a lot of memory per se but that I have so little control over, or insight into, how it's used for my benefit. I've disabled several tag-related features (as you recommended), removed many files from my project, trimmed tagged includes (STL), and reduced cache sizes to no avail. Memory use slowly but surely grows past 1 GB and beyond while editing a fairly small set of files (that's 1 GB of private, dirty memory). If all of my project's source code amounts to less than 100 MB what is that memory being used for? And why does it appear to grow without stopping? (*).
Regards,
Iker
(*) Perhaps it does stop eventually but I restart SE once it hits ~1.5 GB.
-
Does SE provide any diagnostic options/flags that can shed light on memory use?
Iker
-
To try to reproduce the behavior I reported earlier I created a smaller project using source from the tinyxml project (see attached) and ran through a few tests. Here's how it went:
- Open SE with "leak" project. Project "folder view" mode is "directory view". Open tabs for all 6 files in project (see screenshot). Memory ~107 MB.
- Click from one tab to another several times in effort to get SE to populate its caches. Mem at ~110 MB.
- Bring tab for "tinyxmlparser.cpp" to the front.
- Resize the window using lower right handle. Mem grew to 113.94 MB.
- Leave SE alone for ~2 hours (walked away from computer).
- Check memory on return. Memory unchanged at 113.94 MB.
- Go back to resizing window with lower right handle (same tab is still showing). By "resize" I mean left click on the lower-right resize handle, and continuously drag to make the window smaller and bigger.
- Resizing of the SE window reliably increases SE's RAM usage. The size of the increase appears to be proportional to the amount of resize activity. Eg, if a resize for a second or two the increases can be as small as 50K. If I drag around for several seconds I see jumps as high as 8000K.
- Try simply moving the window, without resizing it. Ie, right client on title bar and drag window around. No change in RAM.
- Move an unrelated window in fron of SE and drag it around. No change in RAM.
- Leave SE window size fixed and go back to just cycling through the six open SE tabs (click on first, then second, third, and so on). Many cycles increase RAM by 40-100K, but not all. Note: I'm not clicking on the editor buffers at all - just clicking on the tabs.
- Close all 6 project files with Ctrl-W then reopen them all by double clicking them in the projects window. Repeat many times. Most but not all close-open cycles (of the 6 files) results in an increase of 100-1500K.
- Click on "tinyxmlparser.cpp" tab. Grab scroll bar and scroll to the top of the file and back to the bottom. Repeat many times. Most up-downs increase RAM by 200-300K.
- Click on "tinyxmlparser.cpp" tab. Hit Ctrl-F and search for "TiXmlDeclaration" with option "list all occurrences". I get 4 hits. Repeat exact same search many times. Most of these increase RAM from 4K to 300K. Some leave RAM unchanged.
- Go back to resizing test. Memory continues to grow as before.
- Memory at ~160MB.
- Exit SE. Go through steps above once more. Same results.
- Do one more test... Go to "tinyxmlparser.cpp" then delete one char and hit undo (Ctrl-Z). Repeat delete/undo 10-15 times, check memory, repeat again, check memory, and so on. Each cycle increases memory 500-1000K.
The memory growth here is smaller than in my larger project but appears similar in nature. Attached is a screenshot as well.
I'm running on Xubuntu 16.04. My SE tagging configuration is the same as I reported previously.
Regards,
Iker
-
I'll run through your steps to reproduce later this week and see what comes up. I'm going to run with a default configuration to start out. If I cannot reproduce the leak with that I am going to need to have your user.cfg.xml.
-
Attached is my user.cfg.xml file.
Thanks,
Iker
-
OK. I've spent about a half week on this. Good news and bad news.
1 - Good news) I've identified, found and fixed a few memory leaks. These fixes will be incorporated into the next release.
2 - Bad news - or Jedi mind trick) These are not the leaks you are looking for. It is unlikely that what I found is what brought our SlickEdit memory footprint up to 2G. They were small leaks.
3 - resizing and drawing) Not leaks. We have a fairly significant line drawing cache. You are merely filling up the cache. The test project has a bit over 6000 lines of code, the cache holds 20000 lines before it fills and purges. This is just using memory, not leaking it.
4 - delete and undo) I did not observe a significant memory increase here, and the memory leak tracking software did not identify anything as leaked as a result of this test.
5 - searching) same story, yes, first time use of a feature can uptick memory somewhat, when the form is loaded the first time, etc, but no definitive leaks identified.
6 - segmenting) We have a our own memory allocator for speed. Like any allocator, it can fall victim to some segmenting depending on usage patterns. Some of the memory growth you see could be segmenting, well, it will appear as growth initially, but once there is enough memory in use, the growth will level off entirely. Note, this can happen with the system malloc() too, so there is nothing particularly wrong with our allocator, it's just faster than malloc.
7 - scrolling) see (3)
8 - caches) we have a lot of caches. Tagging especially has a lot of caching going on. It is one of the few things that gives you some limited control over the caches, yet, even there, there are some caches that will just grow as needed. You can't make a setting for everything. I have suggested ways to cut back the tagging cache earlier in this thread.
9 - your config) You were loading a mini-map tool that I did not have access to. Hence, this did not factor into any of my testing.
Hope this helps. I'm still digging around for leaks, as I work. Maybe something will surface eventually.
-
Thanks for the follow up Dennis.
Any way to collect additional memory info during my tests? Eg, are there any diagnostic commands that can print the SE cache sizes and and/or allocator stats? (perhaps via a custom build?)
Thanks again,
Iker
P.S. BTW, the minimap command just toggles the zoom (I didn't use it during my tests). Here's the code:
int minimap_on = 0;
_command void minimap_toggle () name_info(',') {
if (minimap_on) {
wfont_zoom('9');
minimap_on = 0;
}
else {
wfont_zoom('4');
minimap_on = 1;
}
}
-
Sorry, there's nothing built in that would provide any really helpful information. We do have some allocator stats that we occasionally use in debug and unit test builds internally, but they would just provide some stats, nothing of help for tracing down where the memory is going primarily.
-
Any other options I can pursue to help find the problem?
Iker
-
I found another memory leak today. Between Patrick and myself, we have cleaned up quite a few for v23. None of them were showstoppers, but hopefully v23 will run a bit more trim than v22 for you. I would encourage you to participate in the beta for v23 when that time rolls around. That's about all you can do for now.
-
int minimap_on = 0;
_command void minimap_toggle () name_info(',') {
if (minimap_on) {
wfont_zoom('9');
minimap_on = 0;
}
else {
wfont_zoom('4');
minimap_on = 1;
}
}
Heads up. There will very likely be a minimap_toggle command on the view menu in v23. If you define a command with the same name (like above) the original minimap_toggle command will be replaced with yours. You should change the name of yours.
-
Thanks again. Looking forward to the v23 beta.
Iker
P.S. Any chance v23 can include some cache and/or allocator diagnostics to help users provide more detailed reports for issues like these?
-
Don't waste your time trying to work with the VSE people to figure the memory leak out. It has been this way for YEARS (since v13), it wont be fixed. We spent many hours working with them, sending info and responding. It was all a waste of time. All I do is simple editing of files, have a couple small macros that I rarely use. It eats memory like crazy. Working on a 5MB file that was already saved, I hit save again and the memory allocation went up 50MB. I had to shut it down and restart today when it got up to 6.6 GB. Were it not for the fact that I have 100 people who have used it for years (some 15+) I would switch to something else.
So annoying that they don't care to figure this or, or CAN'T.
-
V23 is much better in my experience. V21 or v22 increased slowly and I have also had instances using 6GB, but not anymore in v23.
-
Hi MKJ,
I can assure you, if you're running into 6.6 GB memory leak, and it's simple editing of 5MB files, if you can provide a replicable sample there's likely no reason that we can't fix the issue.
We can't find any reports from you so if you could private message us a case number and the info from Help-->About SlickEdit-->Program Information tab-->Copy to Clipboard we'll get this fixed for you.
Thanks,
SlickEdit Support