duplicated files with date time

[Graeme]

I have a customized version of the save_file function so that whenever I save a file it gets backed up to my google drive.  I recently added some code to make it also make a copy in a OneDrive folder.  For some reason in the OneDrive folder I'm getting a bunch of duplicated files with date and time added to the name.  I have no clue where these files are coming from. Is it possible OneDrive is creating them?  How do I figure this out?

25/02/2018  00:14            12,623 psu.c
24/02/2018  23:52             1,028 psu.c2018-02-24 2351
24/02/2018  23:55            12,014 psu.c2018-02-24 2354
24/02/2018  23:58            12,210 psu.c2018-02-24 2358
24/02/2018  23:59            12,266 psu.c2018-02-24 2359

Code: [Select]

_str save_file(_str filename,_str options)
{
#if 0
   int renumber_flags=numbering_options();
   if (renumber_flags&VSRENUMBER_AUTO) {
      if (renumber_flags&VSRENUMBER_COBOL) {
         renumber_lines(1,6,'0',false,true);
      }
      if (renumber_flags&VSRENUMBER_STD) {
         renumber_lines(73,80,'0',false,true);
      }
   }
#endif
   typeless status=_save_file(options " "_maybe_quote_filename(filename));
   if (!status && file_eq(strip(filename,'B','"'),p_buf_name)) {
      if (p_modified_temp_name!='') {
         _as_removefilename(p_modified_temp_name,true);
         p_modified_temp_name='';
      }
      //_cbsave_filewatch();
#if 1
      call_list('_cbsave_');
      //10:51am 7/3/1997
      //Dan modified for auto-tagging
      if (def_autotag_flags2&AUTOTAG_ON_SAVE) {
         //messageNwait(nls('got here'));
         TagFileOnSave();
      }
#endif
   }
   _str pa = 'C:\Users\GP\Google Drive\slick\copy-on-save\' :+ strip_filename(filename,'DN');
   if (!path_exists(pa)) {
      int result = make_path(pa);
      if (result) {
         _message_box("slick backup make path failed : " :+ result :+ pa);
         return status;
      }
   }

   int result2 = copy_file(filename, pa :+ strip_filename(filename,'P'));

   if (result2 != 0) {
      _message_box("slick backup main google drive copy failed " :+ (_str)(result2));
   }

   int result = copy_file(filename, 'C:\Users\GP\OneDrive\slick-copy-on-save\' :+ strip_filename(filename,'P'));

   if (result != 0) {
      _message_box("slick backup main one drive copy failed " :+ (_str)(result));
   }

   return(status);
}



[jporkkahtc]

Maybe onedrive does this to resolve conflicts?

Saving backups to two different cloud services - are you super paranoid?

[jporkkahtc]

Also, I use "procmon" from System Internals Suite a lot to figure out stuff like this.
Set it to show only "path begins with C:\Users\GP\OneDrive\slick-copy-on-save\"
It will show all file operations in that path, and who is doing them.

[Graeme]

Thanks!  It turned out it is slick auto save that's doing it.  I have it set to a short time - 10 seconds - after slick started occasionally crashing a while ago.  I'm not sure that auto save should be going through save_file but it's easy to fix since I have auto save set to save to a specific folder. 

[Dan]

I certainly don't want to discourage anybody from backing up, but do you use Backup History?

[Graeme]

I certainly don't want to discourage anybody from backing up, but do you use Backup History?

Yep I use backup history a lot.  It's one of slickedit's best features.  I even backup my backup history.

[Dan]

OK... note about Backup History.  You might have noticed for v22 that the format changed.  The date in the XML files with the deltas is the correct time for the file, not the date on the most recent version that is copied in there.  Most users don't look that far under the hood... but it seems like you might

[Graeme]

thanks, yep I knew that it changed because I used to back up the deltas automatically but I can't do that any more.  Now I backup manually using a folder sync tool that comes with exam-diff-pro.

[Dan]

It was a necessary evil with the conversion.

[Graeme]

OK... note about Backup History.  You might have noticed for v22 that the format changed.  The date in the XML files with the deltas is the correct time for the file, not the date on the most recent version that is copied in there.  Most users don't look that far under the hood... but it seems like you might

With one of my projects, I'm getting dozens of copies of each file appearing in my backups folder.  e.g. abc.cs.201  abc.cs.202 ... abc.cs.318  (i.e. 118 copies of the same file with a steadily increasing size).
I have a feeling this shouldn't be happening.  I'm not getting this problem (much) in another project I'm currently working on.  The main difference is that the above files are C# and I sometimes edit them in Visual Studio but mostly in SlickEdit.  Do you have any suggestions for how to find why this is happening.  I'm still using V22.0.0.8.

[Dan]

This absolutely normal.  Here is the change we made:  Before we had to load the whole XML file to get all the versions to list.  So if say an encoding changed or the line endings changed, or some other very large change, the whole delta is written to <filename>.<versionNumber>.  This makes loading the file to get the version list much faster when bringing up the Backup History Dialog.

There was a bug in the original 22 that didn't detect encodings right, so it thought they changed when they did not, and would generate a delta for the whole file.

The threshold for writing a separate file for the delta is in Tools>Options>File Options>Maximum size of individual deltas in archive files.  The default is 10k.  Any delta smaller than that goes in the vsdelta2 file, and thing larger gets written to <filename>.<versionNumber>.

Hope this all makes sense.

[Graeme]

Thanks for explaining.  I don't think I understand all the configuration options for this and it seems to me the help file is a little bit mystifying (e.g. it doesn't explain what a delta is) so I have a suggestion you could consider adding to the help.

<help file begin>
Backup history performance options
When "create backup history on save" is enabled, every time you save a file, SlickEdit determines the difference (referred to as a delta) between the current file and the previous version of the file when it was last saved.  The Backup History Browser allows you to compare any two previous versions of the file or to revert to a previous version etc.  Because the time to generate a delta each time a file is saved can be long for a large file or when large changes are made, SlickEdit provides some options to allow you to minimise the performance hit every time you save a file.  These options allow you to trade off speed against disk space.  To generate the backup history, SlickEdit either calculates the delta (differences) and adds them to the end of the current delta file, or alternatively, to save time, instead of calculating a delta, SlickEdit will make a new copy of the entire file and add the version number to the end of the filename.
<help file end>

I don't understand the option  - Minimum size for fast delta creation (KB) - Minimum size of file to use faster delta creation algorithm. Improves speed of creating backup deltas when saving a file.

What is a "faster delta creation algorithm"?

For this option  -  Maximum size of in-line deltas in archive files (KB) - Approximate maximum size of an in-lined delta. Once the delta is larger than this amount, it is written to an external file.
In the backup options dialog this is called "maximum size of individual deltas in archive files" rather than "in-line deltas"  - with a default setting of 10K.  What is meant by "external file" here  - does it mean a full copy of the entire file is made.

I see that for some of my files there are two "delta" files so it seems there's even more to it than what I wrote above.

[Dan]

Thanks for explaining.  I don't think I understand all the configuration options for this and it seems to me the help file is a little bit mystifying (e.g. it doesn't explain what a delta is) so I have a suggestion you could consider adding to the help.

This could stand to be explained in the help, thanks for pointing that out.

<help file begin>
Backup history performance options
When "create backup history on save" is enabled, every time you save a file, SlickEdit determines the difference (referred to as a delta) between the current file and the previous version of the file when it was last saved.  The Backup History Browser allows you to compare any two previous versions of the file or to revert to a previous version etc.  Because the time to generate a delta each time a file is saved can be long for a large file or when large changes are made, SlickEdit provides some options to allow you to minimise the performance hit every time you save a file.  These options allow you to trade off speed against disk space.  To generate the backup history, SlickEdit either calculates the delta (differences) and adds them to the end of the current delta file, or alternatively, to save time, instead of calculating a delta, SlickEdit will make a new copy of the entire file and add the version number to the end of the filename.
<help file end>

I don't understand the option  - Minimum size for fast delta creation (KB) - Minimum size of file to use faster delta creation algorithm. Improves speed of creating backup deltas when saving a file.

What is a "faster delta creation algorithm"?

Normally when you save we diff the current version of the file (the one you just saved) with the most recent version of the file.  The most recent version used to be stored in the vsdelta file, but now it is stored in the same directory with the vsdelta2 file but has the same name as the file being backed up (test1.cpp for example).  Above the specified threshold (1M by default) we don't perform the full diff, we calculate the differences using a faster method.  We don't use it for everything because it has the potential to create larger deltas.

For this option  -  Maximum size of in-line deltas in archive files (KB) - Approximate maximum size of an in-lined delta. Once the delta is larger than this amount, it is written to an external file.
In the backup options dialog this is called "maximum size of individual deltas in archive files" rather than "in-line deltas"  - with a default setting of 10K.  What is meant by "external file" here  - does it mean a full copy of the entire file is made.

It is not saving entire versions.  Normally the deltas are in the vsdelta2 file.  If the size of the delta itself is above a certain threshold (10k by default), we just add the information about the file to the vsdelta2 file, and store the delta externally, in a file that has the same filename as the original but with the version number appended (test1.cpp.3).  But all that is stored in that file is the delta for that version.  The one caveat is if the encoding or new line characters have changed.  Then the delta is the whole file.  We externalized these files because when we launch the backup history dialog, we have to read the whole vsdelta (now vsdelta2) file to display all the versions and dates.  If there are large deltas, it takes longer for this dialog to launch, even though those deltas are not used until you select that specific version.

I see that for some of my files there are two "delta" files so it seems there's even more to it than what I wrote above.

If you mean that there is a *.vsdelta and a *.vsdelta2 file, this is because the old one was left in place when it was converted.  If you mean that you have more than one numbered file (test1.cpp.3, test1.cpp.15), this just means that more than one delta was above that threshold.

[Graeme]

Once again, thanks for explaining.  I checked a few of the dozens of files that have the version number appended and in all cases the entire file is there so now I realise this is the bug you mentioned  - I'm running 22.0.0.8.  Sometimes I have a file open in slick and another tool e.g. VS and make changes in both.  Hopefully this doesn't mean the encoding or line endings are continually changing.

Regarding the two vsdelta files  - yep, I was wondering why there were two.

Let me know if you'd like me to rewrite what I wrote for the help file.  I think it's kind of complicated to explain but I suspect as it stands there's not quite enough detail for someone to know how and when to adjust the settings  - or maybe they never need changing.

[Graeme]

Is there any way I can fix this bug without upgrading to 22.0.2?