Author Topic: Tagging is lagging.  (Read 2489 times)

jporkkahtc

  • Senior Community Member
  • Posts: 2311
  • Hero Points: 198
  • Text
Tagging is lagging.
« on: May 13, 2013, 07:33:54 pm »
Symbol lookups via Slick tag files is way slower than it should be - often there are really long delays in various symbol operations. For me, it is most commonly on push-ref.

I moved all slick tag files into a ramdisk to try to eliminate disk caching issues.
Perf does seem to improve, but it is still pretty slow in some cases.

First time I tried a push-ref on a particular symbol that I've seen Slick be slow on before, slick was still slow enough for Windows to dim the Slick UI to mark it as unresponsive.

Total size of all tagfiles in use: 93,093,888 bytes.
Tag file cache size (KB): 165536
Tag file cache maximum (KB): 524288


I've found a couple of issues so far using PROCMON to watch my "R:" ram disk.

#1: Slick appears to be reading the tag files constantly even when Slick is in the background and should be idle. I've had slick sitting around in the background for quite a few minutes now, and it is continuing to read my Android.vtg and JDK.vtg files. Seems to hit a page or two per second. What is it doing?

#2: Doing a push-tag causes a lot of activity as you would expect.
About 6,000 events in "procmon" on the tagfiles.
Slick is reading 8 bytes from location 0 of several tag files *MANY* times.
Why? Seems like it shouldn't need to continually re-read the same location in the file.
Still, doesn't seem like enough IO to account for the huge lag times I see sometimes.

#3: Most reading of the tag files is in 8KB chunks.
Presumably, this is because Slick memory-maps these tag files.
Memory mapped IO is slow because it only does IO in chunks of 8KB (or whatever the page size happens to be). Doing a ReadFile( 8mb ) is far far faster than reading the file in 8kb at a time.

I've got 16GB of RAM. I'd be very happy if Slick would simply read all tag files into memory at startup and write changes back to disk later (only if changed?).

Instead of rearchitecting tagfiles, Slick can "warm" the cache by explicitly doing a ReadFile() on the tagfiles to get the tagfile into the memory cache in one go instead of letting the system do it a page at a time.
   void *p = malloc( tagfile.size()); // temporary buffer.
   ReadFile(handle, p,tagfile.size());
   CloseHandle(handle);
   free(p); // throw it away.


#4: Odd file accesses: I only have *.JAVA files open and I'm working with an Android project.
Everytime when I activate Slick it attempts to open <PROJECTFILEPATH>\<PROJECTNAME>_CPP.h - 3 times. I wouldn't expect Slick to be trying to read a project specific *.h file that I haven't told it to, and it is certainly not part of my Java project.

#5: Slick still hangs(permanently) on "File->SaveAs" sometimes.
I had this happen once with Notepad so it is possibly something on my system.
I've had it happen quite a few times with Slick.

Clark

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 5979
  • Hero Points: 467
Re: Tagging is lagging.
« Reply #1 on: May 13, 2013, 10:03:04 pm »
Have you tried the same performance test in V17? Just wondering if there's a difference.

In the future, we do plan to implement much more memory greedy tag databases. For now, too many uses just don't have the memory.