Author Topic: Diffzilla builds file list from excluded directories  (Read 11473 times)

amanison

  • Community Member
  • Posts: 6
  • Hero Points: 0
Diffzilla builds file list from excluded directories
« on: June 11, 2013, 12:08:57 PM »
When doing a recursive diff between two folders, Diffzilla seems to be scanning the files in a directory listed in the "Exclude filespecs" list. Specifically, I see this with .git/. There are loads of files there, none of which are relevant, but the display shows them during the scan. The files don't seem to be compared, but the list building step takes longer than necessary, especially if the filesystem is remote (e.g. mounted with sshfs). It should be possible to skip everything under the matching directory name.

Dan

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 2896
  • Hero Points: 153
Re: Diffzilla builds file list from excluded directories
« Reply #1 on: June 17, 2013, 01:29:20 PM »
Since we can build the list with multiple wildcards, we scan everything, then filter by the multiple wildcards.  So you will see things zoom by that you excluded or don't have anything to do with what you specified.

amanison

  • Community Member
  • Posts: 6
  • Hero Points: 0
Re: Diffzilla builds file list from excluded directories
« Reply #2 on: June 17, 2013, 09:57:01 PM »
I understand the issue. Unfortunately, when working with a remote file system over a VPN, they don't 'zoom by', more like 'stagger by'. In my case, it takes 90 secs just to build the file list. The actual diff goes much quicker. It would be nice if the algorithm were a bit smarter, at least in a simple case, so that remote workflows were more efficient.

Dan

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 2896
  • Hero Points: 153
Re: Diffzilla builds file list from excluded directories
« Reply #3 on: June 18, 2013, 01:23:20 AM »
We studied the problem pretty thoroughly when we switched to using this method.  Since we support multiple filespecs, we would have to do the list for each wildcard.  It is usually faster to list everything and then filter the list.

Clark

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 6823
  • Hero Points: 526
Re: Diffzilla builds file list from excluded directories
« Reply #4 on: August 28, 2013, 05:10:02 PM »
The plan for v19 is to have completely new code for traversing trees of files and specifying excludes. Directories that are excluded will NOT be traversed and only one pass will be necessary even if multiple wildcards and excludes are specified. Also, v19 will support a super set of ant syntax. That is, you will be able to write "**/backup/**" to exclude a directory, or you can write "backup/" to exclude a directory. You will also be able to specify wildcards like the "**/somedir/t*/*.cpp".

chrisant

  • Senior Community Member
  • Posts: 1410
  • Hero Points: 131
Re: Diffzilla builds file list from excluded directories
« Reply #5 on: August 29, 2013, 08:09:22 AM »
Oh yay!  8)
Out of curiosity, if you know yet:  do you expect insert_file_list() will be extended, or will it be a new function?
I don't really care which, I'm just excited about a tree traversal routine that can skip top level directories that are excluded -- among other things, the sync_project_with_scm macro in the forums here can benefit from that.

Clark

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 6823
  • Hero Points: 526
Re: Diffzilla builds file list from excluded directories
« Reply #6 on: August 29, 2013, 04:56:33 PM »
Yes, insert_file_list will have this new support .While you can be lazy and things will almost always work. There are scenarios where it is best to specify a directory followed by the wildcards as separate arguments. Something like this:

insert_file_list(' <options> c:\somdir\ -wc dir\t*\*.cpp whatever\*.h -exclude backup\ **\somedir\t*\*.h');


Our updated macros will do this instead of mashing the directory and wildcard together which will give incorrect results due to internal guessing that has to be done.