Author Topic: B3: SE Freeze with Smart Open, "Date Modified" and many files with similar names  (Read 2869 times)

Dan

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 2373
  • Hero Points: 133
Just discovered an interesting thing when opening my second instance.

I enabled "Date Modified", but I didn't choose to sort on it. Then opened my project and typed a key into the edit box. SE did NOT freeze.

After sorting on "Date Modified", then typing a key, then SE started freezing (I will wait for it)

So the issue is also related to whether "Date Modified" column is chosen for sort.

You can't get it in a local directory though, right?

rowbearto

  • Senior Community Member
  • Posts: 1483
  • Hero Points: 113
My 2nd instance of V22 did eventually unfreeze. Then after unfreezing, didn't freeze when I typed new letter into smart open. But when I tried the main.cc thing, sorted on "Date modified" and then clicked on main.cc, it froze again (and then eventually unfroze). V21 eventually unfroze as well.

My main projects all have files on the remote file system because that is where all of our source code is stored. I am not able to use a local file system for my main work as that is not how clearcase works. Also, our home directories are on remote file systems (different than clearcase file system), so my projects that don't use the clearcase file system (very small python projects) are on my home file system, which is also not local (but not prone to slow issues).

Nevertheless when I use these small python projects from my home dir, I don't seem to have this type of freezing.

Dan

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 2373
  • Hero Points: 133
The fix here is probably going to be to put a timeout on getting the dates.

rowbearto

  • Senior Community Member
  • Posts: 1483
  • Hero Points: 113
Not sure that is such a good fix, because I want the dates so that I can sort the files in smart open.

I'm not fully convinced it is due to remote file system, because why do I get no freeze when not sorting by Modified date? Only time I freeze is when sorting by Modified date. When not sorting by modified date, you are still showing a timestamp, so what timestamp are you showing? Are you scanning the timestamp?

Why does this depend on sorting by modified date? If it was due to remote file system, I should also freeze when sorting by name.

rowbearto

  • Senior Community Member
  • Posts: 1483
  • Hero Points: 113
Besides the question of why only the freeze when sorting by Date Modified, there is another aspect to this problem.

In the workspace that I sent you, there are a total of 31625 files.

So if I type 1 letter in smart open, even with a local file system, the number of files containing that letter will be very large. Will SE be scanning for the dates of this large subset of files before allowing a 2nd character to be typed in? Even with a local file system, scanning for the dates of up to 31625 files can take a very long time.

I think the Smart Open should be scanning the dates in the background to allow to type in multiple characters, chances are I'm going to type multiple characters, so why waste CPU and waiting time scanning files containing 1 character when another character will surely be typed in?

Also, I have only 56 files that match "main.cc" out of the 31625 total in the workspace. I wrote a script to perform "ls -al" on ONLY these 56 files, it takes less than 1 second, even with all these 56 files on the remote file system. So there is no reason Smart Open should be freezing when I do: 1. Smart Open without "Date modified" column, 2. type "main.cc", 3. Turn on "Date Modified", 4. Sort by "Date Modifed", 5. Click on one of the main.cc, 6. SE freezes. Why is SE freezing when it takes only 1 second to scan these 56 files?

So I think the Smart Open is performing a date scan on many many more files than needed, especially as each character is typed in, but also when only the 56 main.cc files are displayed, I suspect SE is scanning dates for much more than these 56 (even with main.cc in the edit box), and doing this in the foreground freezing SE.

But also, even when just the 56 files match the main.cc and are displayed in the smart open tool window, I suspect that SE is performing a date scan on many more than just these 56 files when I click on one of the main.cc, SE opens this main.cc, and then I think SE is scanning for the date on many more files than just these 56.
« Last Edit: September 12, 2017, 01:59:15 am by rowbearto »

Dan

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 2373
  • Hero Points: 133
I'm really sorry, but we can't get to this for 22.0.0.  The underlying code is fairly complicated and involves a class that caches file lists that is used in multiple places.  So we can't risk making a mistake in it at this late date.  We will get to it, but we can't do it for this version.

rowbearto

  • Senior Community Member
  • Posts: 1483
  • Hero Points: 113
OK. So the freeze is due to scanning more files than required? You understand now the root cause?

Dan

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 2373
  • Hero Points: 133
Yes.  It's just that it's a V/M/C sort of layout, so to change it to just refresh what is on the screen, especially in a hurry, is a formula for disaster. 

But then what this business has taught me is that making changes to existing code, particularly complicated code, in a hurry, is a formula for disaster.  I try not to check anything in after 4pm on Fridays ;)

rowbearto

  • Senior Community Member
  • Posts: 1483
  • Hero Points: 113
Hey I hear you, I've learned that lesson too! I'm fine with waiting for a quality fix. Thanks for looking into this!