Macros are broken (24.0.0.8, hotfix 3)

[sigmund]

Steps to reproduce:

1. Make 25 copies of a line with the content x;y;z;w
2. Move the cursor to the first of those lines
3. Record a simple macro like this one (copied from C:\Users\sigmund\Documents\My SlickEdit Config\24.0.0\lastmac.e):

Code: [Select]

#include "slick.sh"
_command last_recorded_macro() name_info(','VSARG2_MARK|VSARG2_REQUIRES_EDITORCTL)
{
    _macro('R',1);
    end_line();
    prev_word();
    prev_word();
    prev_word();
    copy_word();
    end_line();
    keyin(";");
    paste();
    keyin(".test");
    cursor_right();
}

4. Run the macro 20 times (repeat-key 20 Ctrl-12).
5. The result is not as expected:

Code: [Select]

x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;y.test
x;y;z;w;
x;y;z;w
x;y;z;w
x;y;z;w
x;y;z;w
x;y;z;w
x;y;z;w
x;y;z;w
x;y;z;w

When just keeping Ctrl+F12 pressed for a few seconds the output can get even more corrupt, here is the result of a similarly simple macro that involved a find-next used to find the next semicolon:

Code: [Select]

a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;c.test
a;b;c;
a;;
a.test;b;c;
a;:CHAR 2 * 11 0 [] 0.test;b;c;c.test
a;b;c;c.test
a;b;c;c.test

Running macros is also very slow, making them almost unusable for manipulating more than say a few hundred lines.

Here is my output from Help -> About SlickEdit:

Code: [Select]

SlickEdit Pro 2019 (v24.0.0.8 32-bit)

Serial number: WB943076
Licensed number of users: Single user
License file: C:\ProgramData\slickedit\24\slickedit.lic

Build Date: October 2, 2019
Emulation: CUA

OS: Windows 10 x64
OS Version: 10.00.0 
Memory: 57% Load, 9401MB/16292MB Physical, 18529MB/54712MB Page File, 573MB/4095MB Virtual
Shell Information: C:\tools\JPSoft\TCMD25\tcc.exe /LD /LH
Screen Size: 2560 x 1440

Project Type: Other
Language: .txt (Plain Text)
Encoding: Automatic

Installation Directory: C:\Program Files (x86)\SlickEdit Pro 24.0.0\ (non-removable drive,NTFS,140221MB free)
Configuration Directory: C:\Users\sigmund\Documents\My SlickEdit Config\24.0.0\ (non-removable drive,NTFS,140221MB free)
Migrated from: C:\Users\sigmund\Documents\My SlickEdit Config\23.0.2\


[Clark]

We will look into this. Looks like something is going wrong with the clipboard.

Recorded macros are very inefficient. A custom macro would run WAAAAAAY faster. Although, not typically worth it to hand write unless you're operating on a ton of lines.

[sigmund]

Thanks!  Recorded macros are often very convenient so it would be nice if they at least were somewhat efficient, especially in combination with repeat-key.

[Clark]

So far I'm not reproducing this. At least not without writing another application which leaves the clipboard open so SlickEdit can't create clipboards.

If you use repeat-key for 1000 iterations, how long does this take for you.

For me, it takes about 3 seconds. My machine is pretty slow. I'm running Windows in a VM and it is a VERY slow machine.

I ask this because given your comment, your performance may be much worse possibly due to the fact that this macro copies something to the clipboard for each iteration. This could indicate a more specialized problem. Maybe some application is running in the background constantly opening the clipboard to fetch results that have been copied to the clipboard by SlickEdit. This could be causing both the performance problem and the failure problem.

[sigmund]

Yes, both the correctness and speed issues are related to updating the content of the clipboard.

I had Windows clipboard history enabled (but not "clipboard sync"):
https://www.howtogeek.com/351978/using-windows-10s-new-clipboard-history-and-cloud-sync/

However, it is still broken and slow after I disable this.  If I run a simple macro that only copies from the clipboard, then a repeat-key with 1000 iterations takes less than a second.  However, if the macro copies to the clipboard, then it produces incorrect results as shown above.  (It not slower in this case, but it only updates about 30 of the 1000 lines.  If I instead keep Ctrl+F12 pressed, then I see that it runs much slower.)

I have tested this on two different machines (Windows version 1809) and they both have the same issue.

[Clark]

I’ve completely rewritten the low level clipboard code for windows. Hopefully  this new code will fix these issues. The new code made this macro run twice as fast (half the time). It will definitely have an effect.

Support will give you directions for installing this new vsapi.dll (assuming it passes more tests).

[hs2]

Sounds great Clark ! Performance matters

[SlickEdit Support]

PM sent.

Best,
SlickEdit Support

[sigmund]

Excellent, thank you!  I just tried the new vsapi.dll with the test macro that I posted, and now it works as expected.

It used 13 seconds to run with repeat-key and 1000 iterations.  With Windows clipboard history enabled it used 16 seconds (but each line is the same, so there is no more than a single update to the history).

[Clark]

Sounds like it fixed the flakiness problem. The Qt clipboard code uses some sort of OLE APIs which are more prone to blocking failures due to another application having the clipboard open. The new code I wrote uses simple Win32 calls.

I’m still surprised it takes so long to run. I l’ll do some testing on other machines and try the windows clipboard history option.

[sigmund]

It most certainly looks like the incorrect behavior is gone

Right after a reboot the test that used to take 16 seconds ran in about one second.  And now 10 minutes later it takes about two seconds.

[Clark]

Awesome! Glad this works way better.

[sigmund]

Unfortunately there is a bug in the new copy-from-clipboard code.  Depending on where I copy from, in certain cases there is a large number of spaces appended to the pasted result (due to some kind of buffer overrun?).  This happens when I for instance copy text from a .NET System.Windows.Forms.MessageBox window.  It does not happen when I paste the same clipboard content to e.g. Notepad.

(I have sent a more detailed report to SlickEdit Support, but I have not heard back from them.)

[Clark]

24.0.1 has completely rewritten clipboard code for Windows. This was done to fix instability when using Qt's clipboard code on Windows.

Not hot fixable. Fixed for 24.0.2. Added code to truncate at the first null.

Being able to support binary clipboards on Windows would have been nice but it's not necessary and no other platform supports it either. v24.0.1 has some new hex clipboard features for supporting binary clipboards. In hex mode, you can copy/paste binary clipboards within the same instance of SlickEdit either as hex bytes (left hand side) or binary text (right hand side - definitely supports nulls). This is easily possible because SlickEdit will detect that an internal clipboard is being used and not copy text from the system clipboard.

[sigmund]

Seeing that 24.0.1 was just released, would it be possible to get a link to a vsapi.dll with the fix?  The problem occurs whenever I copy something from the Take Command (https://jpsoft.com/) console window..