Author Topic: feedback on ver25 with forth awareness support  (Read 1209 times)

johntse

  • Junior Community Member
  • Posts: 4
  • Hero Points: 0
feedback on ver25 with forth awareness support
« on: July 22, 2020, 06:50:16 AM »
Anyway, some initial impression

Wow, immediately after installation
My forth code is properly color-coded

Some simple question but very important questions, especially qn2 n qn3
1)   How do I do program tabs and also with spaces
2)   How to have keywords with different colors, this is because there are classes of keywords, example stack manipulation keywords – one color, logic operation keywords another color etc, if you look at the sample color-coded file I send you, you will see that different class of keywords have different colors
3)   How to set autocomplete for user words because in forth, a lot of user-defined constants and variables, example: constants =  eCMD_TRANSCEIVE or eCMDReg
Or variables = bAUTHCTR or bSAMCSN7, too long to type in all the time. So autocomplete really useful

The good thing in forth is that your editor can be programmed with the scripting language to recognize a constant or variable and in forth, a lot of such formatting is standard, so the editor can really be forth aware easily and not only can prompt auto-complete, should be able to also give these constant or variable a distinct color as well.

In fact, if I am not mistaken, the editor is already doing this by recognizing that after : is a user-defined word function and give it a distinct color. This alone is one capability that I have never seen in any editor. Of course, every editor can claim that they can do it. Still only in Slickedit, I have seen it done now.

I am not even aware that it is possible to do this because of the two versions of programmer editor I use cannot do this easily without extensive programming

So to add for Example,  forth rule = any complete word after a EQU or a CONSTANT is a constant, any complete word after a BUFFER or a  xVariable is a variable, where x = c or w or l or none just remember that in forth space is not important,  that is why my example file, aligning all the code for easy reading is more important due to more space is the same as one space.

So  cVariable “ one space “   thisvariable  is the same as cVariable  “ lots of space “ thisvariable             

In fact, my code can be easily factored to lot smaller. Aligning all of it shows where factorization is the most effective.
That is how forth programmers write forth code, they do the long version first, space aligning everything properly and in the proper color, then looking at the entire code page,
They can immediately notice where to factor repeated code and keywords and shrink the code as much as 50-70% without making the code unreadable

So at one stroke, the editor not only can prompt the correct word and also able to display the whole line including comments, the first time I have ever seen this capability in forth editor

Wow, all user-defined words appeared on search tab, super zoom-in function, awesome…

Pls show me the script file that I can look at and try to add such functionality, if possible do the above two examples or how did SlickEdit can recognize the user-defined word function. So that I can reference it and learn to add more. this will make the editor super duper for sure.

Since forth uses mostly user-defined words in constants, variables, word function other than the official keywords, etc. if the editor is aware of it, then the editor practically almost write itself after 3 letters

Forth is really a simple language with very friendly structure and very few rules which is it’s a blessing and also it’s a curse.

I am practically drooling already.
john

patrick

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 1818
  • Hero Points: 151
Re: feedback on ver25 with forth awareness support
« Reply #1 on: July 22, 2020, 01:56:41 PM »
1) For using tabs and space, with a Forth file open, go to Document -> Forth Options, and then go to the Formatting entry, and turn on indent with tabs, and then hit the tab key for tabs, and space for spaces.  SlickEdit generally tries to use tabs for leading indents and spaces for everything else, but since there aren't really any syntax expansions for forth, I don't think we'll interfere with the mix of tabs and spaces you put in.

2) If you go the Color Coding entry under Forth Options, and click on the token tab to see how it's set up.  Right now everything that's not a number or a comment is classified as a keyword.  You can take sets of words and classify them as one of the other existing token types, like "library symbol" or "operator", but you can't create new token types.  The colors of the token types can be set in Tools -> Options -> Appearance -> Symbol Coloring.  The Symbol Coloring is global for all languages, each language defines what tokens get mapped to the different token types.

3) For Forth, anything that's defined in the color coding tokens is eligible for completion. There's also a function called "forth_proc_search" in YOUR_INSTALL_DIR/macros/ccode.e.   It tags function names and vars based on some regexes.  It's the simplest type of tagging function we can do, so you can see what words it treats as creating words.  It can't do anything fancy like recognizing a word you've defined has CREATE in it, and then infer your word should now be a creation word.  You can quickly see what words in a file are being tagged by bringing up the "Defs" tool window (View -> Tool Windows -> Defs if it isn't already visible).

We don't have EQU defined.  Hadn't seen c/wVariable before.  Since projects will have a lot of custom creation words, maybe we should tweak the code for the next beta release so you can specify local ones conveniently by doing set-var like "set-var local-forth-defining-words cVariable wVariable BUFFER" and not have to change the code.  There's nothing wrong with changing the code, but you do end up having to merge in changes whenever there's a hotfix that's changed one of the files you changed, which can be a pain.

Clark

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 6826
  • Hero Points: 526
Re: feedback on ver25 with forth awareness support
« Reply #2 on: July 22, 2020, 03:27:23 PM »
Some of this is redundant since Patrick has already responded.

Thank you so much for thoroughly checking out the new Forth support. We aren't Forth experts here so we can really use your help.

1) How do I do program tabs and also with spaces

Do you mean configure tabs and indenting?

Tools>Options>Languages>Miscellaneous Languages>Forth>Formatting

If current file is a Forth file, this is quicker:

Document>Forth Options>Formatting

2) How to have keywords with different colors?

Right now, it's not great. You can use one of the pre-defined categories that are already defined (operator, punctuation). However, this might not work well because if you change the color for that category, it will effect all languages using that category.

What's done for some languages, like Markdown, is extra language specific color categories have been added.

(Example) We added the following specifically for Markdown:

Makrdown Blockquote
Markdown Bullet
Markdown Code
Markdown Emphasis
etc.

Users currently can't add these (bummer I know). If you can advise us what extra color categories would make sense for Forth, we should be able to add them. If you want constants, variables, and functions in a different color, you will need to use symbol coloring. It's an extension to color coding.

Note that currently the tagging that has been added for Forth doesn't support local variables for scoping yet. I don't think there will be time to add this especially since we haven't fully figured out the syntax.

From the SlickEdit command line. Type "fp forth_proc_search". This will display the source code for the very simple Slick-C function we wrote to find symbols in forth. This type of parsing solution is pretty much never used for supporting scoping for things like local variables. Users do sometimes write these _proc_search functions since they aren't that complicated. More advanced parsing (WAY more complicated) is done in C++ and typically built-in to SlickEdit (i.e C++ parser, Python parser, there are boat loads of them). They take way more time to write. The windows version of SlickEdit supports DLL add-ons. However, the currently the macOS and Linux version do not support this. It's realistically too much work for customers anyway.


3) How to set autocomplete for user words.

The easiest way to do this is to create a source file with the variables and things you wanted pre-defined, create a language specific tag file, and add that source file to it.

SlickEdit already ships with a built-ins file for forth. If you want to take a look at it, type "e plugin://com_slickedit.base/sysconfig/tagging/builtins/builtins.forth" on the SlickEdit command line. This interesting command ends up editing builts.forth directly out of a zip file we ship (com_slickedit.base.zip). SlickEdit automatically creates a Forth language specific tag file for this file.

You can add your own language specific tag files for Forth here:

Tools>Options>Languages>Miscellaneous Languages>Forth>Tag Files

Hit the plus sign to add a another tag file. It will prompt for the tag file name and source file(s). You can add/remove files from your tag file later as well.

4) Coloring words after a colon in function color.

Here's the SlickEdit (not Perl syntax) regex which does this trick:

(#<=([^\x0-\x8\xB\xC\xE-\x1F\x21-\xFFFF]|^)(\:|variable|fvariable|constant|fconstant|2variable|2constant|see|forget))[ \t]#[\x0-\x8\xB\xC\xE-\x1F\x21-\xFFFF]#

SlickEdit's regex engine supports variable length look behind. NO OTHER product supports variable length look behind. This part "[ \t]#[\x0-\x8\xB\xC\xE-\x1F\x21-\xFFFF]#" searches for ANY forth identifier. This complicated expression below matches something specific before it:
 
(#<=([^\x0-\x8\xB\xC\xE-\x1F\x21-\xFFFF]|^)(\:|variable|fvariable|constant|fconstant|2variable|2constant|see|forget))
 
This is a little crude but it gets the job done. In the future, SlickEdit will have a better way to do this. Currently, the entire match needs to be colored in one category. Look behind text (or look ahead text -- not done here) isn't considered part of the match. In the future, you will be able to specify different colors for regex match groups and/or string offset/length.

You can use a Perl syntax regex instead. It's not much different. Replace #<= with ?<=. Replace # with +.
« Last Edit: July 22, 2020, 07:01:06 PM by Clark »