Where are the functional languages?

[divisortheory]

I'd been eagerly awaiting the release of SlickEdit 2009, hoping that it would finally support AT LEAST ONE functional language.  There's nothing.  No ML, no Haskell, no Lisp, no F#.  With the exception of F# all of these languages have been around for DECADES and tons of people use them, but not an ounce of support, but somehow D is supported?

Let's take Lisp for instance.  Parsing Lisp code is trivial.  There's got to be something I'm missing here.  Haskell?  The compilers are open source and support a foreign function interface, so you can access the parser, the metadata, the type information, everything from C, .NET or whatever language SlickEdit is written in.  All it takes is one Haskell programmer who knows what he's doing on the team and this would be done in a very short time. 

Even Microsoft has thrown their support behind F# recently and it will be shipped in Visual Studio 2010 with first class support.  It's obvious that functional languages are going to be major players in the coming years with the increase in importance of parallelization and massively multicore processors, so why deny them for such an insanely long time?

[Matthew]

I've been trying to think the perfect response to your post all day, but I haven't come up with it yet. So I'm going to just start typing and see where it goes...

First off, I'll say that yours is the first real impassioned plea for functional language support I've seen. There may have been others, I'm just saying it's the first I've seen. But impassioned pleas are what we need. Features get pushed to the front of the line for several reasons, but customer demand (esp. on the forums) is chief among them. Coincidentally, while manning our booth at GDC (Game Developers Conference) last week I had one visitor ask about Haskell and another mention Scheme.

I think this can be summed up with "What if SlickEdit threw an Erlang party and nobody came?". We're very much into support for 'smaller' languages, and we try to be proactive with many of our features, but language support for smaller languages pretty much requires us to gauge what folks are asking for.

All it takes is one Haskell programmer who knows what he's doing on the team and this would be done in a very short time.

As far as I can tell, that person does not exist in this building. However, there are a couple of us having brief dalliances with F#.

It would be fairly quick to add support for several functional languages in a short amount of time, but it would be superficial support at first. By that I mean basic sytax/keyword coloring and perhaps some aliases and syntax expansions. Fuller support, namely context tagging, requires a lot of Slick-C code and/or a full-blown parser.

And I understand the point you make with F#, with Microsoft productizing it VS2010. But I'm not 100% convinced that it'll become the tool of choice for parallel programming in .NET since these libraries will be available to C# users, and VC++ developers will be getting the PPL as well. Now that alone won't drive language support decisions we make, but it is a factor.

So now that between the two of us we've named 6 functional languages, how would you rank them? And are we missing any big ones?

[divisortheory]

After I posted my original message, I re-read it and realized that it was probably harsher than it should have been.  So thanks for responding anyway

And I'm not surprised that there's no Haskell programmer in the building   But it has a very active community, I doubt it would be hard to find someone, even if it were only a contract basis, and even if they were only telecommuting, that could (and would love to) do it.  (And if you feel like private messaging me, I actually know an amazingly smart person who might even be interested in a short contract such as that).

It's difficult to rank them, because we really don't know what will happen with F#.  But here's a few thoughts:

If you implement support for F#, you've already done 95% of the work to implement support for Ocaml, and vice versa. 

Same goes for Lisp and Scheme.  Lisp and Scheme are probably the easiest, because parsing Lisp is so easy.  There's an entire open source Lisp parser written in Haskell that occupies maybe 20-30 lines of code.  See 2 paragraphs down for a caveat about Lisp, however.

My initial guess is that F# will have the quickest short term adoption, specifically because it will have so much backing as well as a great IDE, and interoperate well with other .NET languages.  On the other hand, if it already has a great IDE this might cause you to wonder why you should bother with it.  But on still the other hand, if you have a large C# or VB.NET customer base, then there's no reason to suspect that your F# customer base wouldn't at least have a similar ratio.

What happens in the long term remains to be seen.  Lisp and Scheme almost certainly aren't going to emerge as the winners.  In addition, there are a number of IDEs available for serious Lisp programmers, and many of the rest use Emacs because they can program the actual editor in their favorite language.  So I wouldn't bet my money on Lisp for bringing a lot of people over to SlickEdit, although like I said it should be really really easy, if you're ever looking to do it just for the sake of completeness.

Haskell has the most theoretical potential to be the language of choice for mainstream parallel programming due to the fact that it's one of the few purely functional languages available, and it's by the far the most widely used purely function language.  But it is pretty advanced and basically just going to be too difficult for a lot of people.  However, one of the original creators of the Haskell language, and the principal designer of the most widely used Haskell compiler is employed by Microsoft Research, as is the other lead developer on that compiler, and in fact the development of the compiler is actually funded in large part by Microsoft Research.  This fact alone suggests that perhaps Microsoft has plans for it.  It would make sense, there's two primary .NET imperative languages: the "basic" one (VB.NET) and the "advanced" one (C#).  Now that they have F#, they're only missing the advanced functional .NET compiler.  I don't see it happening in the next 2 years, but early bird gets the worm.  And of course this is all speculation. 

I really don't know anything about Erlang, so I can't comment on that actually. 

At the very least, if you had Haskell support you'd get a minimum of 1 sale out of it (me).  There is literally nothing else except for Emacs and Vim available, and a few IDEs that are in early alpha stages but need major polishing and work before they're ready for prime time.  The early reception of these early alpha demos have been very positive though.  Would people shell out money for the entire IDE though just to get someone they can use a free alpha version of, or Emacs or Vim?  I don't know, but if you have data on how many of your customers use SlickEdit for C++, Java, or for that matter any language that Eclipse has a plugin for, then that might give you some ideas.  I can say that there is definitely some serious demand for a powerful and polished editor, and the time to market for SlickEdit would probably be far less than the others, even if they're demoing alphas, since you already have the entire rest of the product, and all the polish, framework, and everything else is already there.

Another crazy idea might be to make a freely downloadable SlickEdit Shell, similar to the Visual Studio Shell, that supports nothing except for editing plaintext files.  Then sell the "functional language pack" for a smaller price, and perhaps this model of selling languages packs could be extended to other subsets of languages as well. 

If nothing else, it would be a huge step just getting syntax coloring and automatic indentation in for the languages.  And I don't even think you'd need a serious Haskell programmer for that (indentation in Haskell isn't trivial, however).  At least that way people wouldn't have to actually switch to another editor to do anything in the language.  Another easy but great feature is an option to have the editor automatically convert certain short strings into "prettified" versions.  For example an expression (\ ... ) in Haskell is an anonymous function (lambda expression).  The \ is the lambda.  When you encounter such an expression you could (if the option is set) replace the \ with an actual unicode lambda symbol.  Right and left arrows -> and <- are also used very often, those could be replaced as well.  Some of the basic editors out there for Haskell already do things like this, and people have hacked their VIs and Emacses to do similar things. 

Eventually for full-fledged cream of the crop support,  you'd want to support Vim and Emacs keybindings to woo the hardcore, and all the other goodies that SlickEdit usually supports like mouse over tooltips with type inference, statement completion, intellisense, etc. 

But I'm getting ahead of myself.  The point is that really just anything other than Emacs or Vim would be nice, even if it doesn't have many of the advanced features.  Limited support such as that may not bring in a ton of new customers just for that, but at the same time it shouldn't require a lot of man hours either, and it might bring in more comments, interest, and feedback to allow you to make better decisions. 

Edit: I also have always been a believer that having good tools will increase the number of people wanting (willing) to learn / program in a language.  There might be people out there who are brilliant, but just feel like Vim and Emacs are painful, but they'd love to program in any of these languages if the tools were better.  And like I mentioned earlier, the functional revolution is coming.  Just wait until cloud computing starts becoming more prevalent   Google once said that their MapReduce algorithm, which is pretty much the core of everything, is impossible without functional programming due to the massive parallelization required.  Cloud computing is really pretty similar in its computational requirements.

[jbhurst]

According to the TIOBE index (http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html), functional languages are in pretty good demand.

Perhaps surprisingly, LISP/Scheme is currently at position 23, the best of any of the functional languages. Erlang is at position 31, and Haskell is at 34. Scala, the leading functional language for the JVM, is at position 35. F# doesn't get a mention.

I would guess that Erlang will continue to see big gains, and Scala also will grow this year. (I've been getting into Scala myself!) F# also will be big once it ships in Visual Studio. These are the (functional) languages with commercial buzz at the moment.

Regards

John Hurst
Wellington, New Zealand

[divisortheory]

To be fair, F# is still considered a technology preview and not even in alpha yet, so it doesn't surprise me that it doesn't rank yet.  Interesting to see Lisp so high.  I'd be surprised if F# isn't bumped to the low 20s within the first two years of its release. 

After my post I realized that I also forget to mention Clojure.

[zammazingo]

I have asked for Lisp support couple of times which should be compatible with "Slime". I could not get much support for that idea at the time.

If possible I would like to see some support for functional programming languages in SlickEdit.
Thanks a lot.

[zack]

I would also cast a strong vote for support of functional programming languages.  My preferred list in order is Erlang, Scala, and Haskell.  I assume many LISP users prefer Emacs and I'm not really a Microsoft person so I don't use F#. 

A full parser for these languages would be great, but simple keyword and string syntax highlighting would also be extremely helpful. 

Thanks,
Zack

[divisortheory]

Late response, sorry but I've been pretty busy at work.  My preferred order would probably be Haskell -> Erlang -> anything, as most other languages at least have some sort of alternative available (not sure about Scala).  After thinking about Haskell support some more, I realized it would actually be really easy to get basic Haskell parsing in, even without a Haskell programmer on board to make use of the compiler source.  The reason is that you can specify optional type signatures for functions.  It shouldn't be that bad to look for these and then only support parameter info / completion on functions that had these.  It'd be something anyway, and almost 90% of code has these anyway since it's good form.  There's a few issues regarding scope, shadowing, and order of declarations, but nothing that hasn't probably already been addressed with other languages.

That, indentation, syntax highlighting would probably sell me, even without compiler integration.  Source candy (displaying certain character sequences such as \, <-, and -> with unicode symbols such as λ, ←, and → while not actually modifying the source on disk) would make it even more irresistable, and also probably be easy.

[witness9]

I will add another vote for Scheme support as well.  I don't recall if I ever asked SlickEdit for support when I was working a lot with Scheme a few years ago.  I easily found a color-coding extension for TextPad and went with that, but would really have preferred to use SlickEdit.

[Dennis]

If you implement support for F#, you've already done 95% of the work to implement support for Ocaml, and vice versa.  Same goes for Lisp and Scheme.  Lisp and Scheme are probably the easiest, because parsing Lisp is so easy.  There's an entire open source Lisp parser written in Haskell that occupies maybe 20-30 lines of code.  See 2 paragraphs down for a caveat about Lisp, however.

Yes and no, with respect to parsing Lisp and Scheme and other functional languages.  Yes, they do typically have very, very compact lexicographic and syntactic models.  However, the kind of parsing SlickEdit does isn't just putting together a parse tree -- that alone is actually not all that useful.  Useful support comes from scraping from that parse tree symbol information, such as where a function or class is defined, parameter lists, and, the hardest part, what S-expressions containing lists of words are data and what are effectively strings.  Some dialects make this easy -- traditional lisp did not.  Also, when you consider how common it is to use second-order functions (functions which 'defun' other functions), it becomes considerably difficult to garner any real information from the code that will assist you in writing Lisp.

Yes, I agree, we could provide color coding and syntax indent and some other basic editing features for these languages with a certain amount of effort, but the difficulty level for implementing these features unfortunately does not correlate with the fact that you can write a scheme parser in 20-30 lines of scheme.

So, in summary, it's harder than it looks to provide good language support.  Whether or not users would get excited about just having syntax coloring and maybe some paren matching and other default behaviors remains an open question for the forum.

I hope I did not sound too harsh either.  I would like us to have time to implement more support for many languages.  It would help us gain popularity in educational settings, I'd love to see SlickEdit used by students in a Comparitive Programming Languages class.

[divisortheory]

In all fairness, you sounded less harsh than I did in my original post   And yes, you're right on all the points regarding lisp, although I'm not able to think off the top of my head of what kinds of difficulties local function declarations would introduce (although admittedly it's been a long time since I've done anything in lisp).  You have a cheap symbol table built up with scope / symbol information, and the function declaration just introduces a new scope which shadows previously defined declarations.  Or is there something else to it in lisp?

Back to the Haskell example since it's the language I'm the most familiar with, a majority of the time you have something like this (function is silly but simple at least):

Code: [Select]

testFunc :: Int -> Int -> String -> Bool
testFunc x y str =
     z == str
     where z = (read str) + 9

The signature of the function is printed right there (even though it's optional it's almost always there, especially in core libraries where it's very rare not to have it), so showing parameter type information is pretty much done if you have a parse tree.  Arguments are Int, Int, String, return value is Bool.  Same goes for local functions, as the signature can be (optionally) specified on a local function too.  If it's not there just don't display parameter info for that function, or display as much as you can with some really basic type inference (e.g. it's obvious that z is a string since it's being compared for equality with another string). 

This is actually probably even more useful in Haskell than it is in other languages.  The reason is that in most languages when you look at the function signature you see the parameter names and types interleaved with each other, for example void f(int count, bool check_for_duplicates) so when you're looking at it it only takes a quick glance to figure out everything you can possibly deduce from the arguments, their names, and their types.  In Haskell it's not that simple, for starters because the signature can occur ANYWHERE in the file, before or after the actual definition, doesn't matter.  So even if you browse your way back to the function definition or signature to look at the parameter names or types to get some clue how to use them, you have to correlate the names with the types, which even in the best case scenario means you have to do pairwise matching on two lists of strings in your head, and in the worst case scenario means you have to browse around to different parts of the file searching for the other half.  Parameter lists can get pretty long too, especially when you bring higher order functions into the picture.  So actually this feature alone is really, really useful.

Highlighting numeric literals, strings, reserved words, and displaying basic type information even just in the easy case is already leaps and bounds above what's available (and production quality).  Add automatic indentation and you're gold.  Indentation is slightly more difficult, but not horribly, there are only a few odd corner cases. 

And while it's true that most functional languages see their primary use in an academic setting, they are becoming major contenders in areas such as data mining and search engine development, text analysis and other like fields, which despite being big players even today, are still in their infancy and ready to explode, especially as cloud computing matures.  And on top of that we're getting pretty close to having the average home user's workstation contain 8, 16, or possibly more cores, which has real potential to be a game changer.  (Do I sound like an evangelist for functional programming languages yet?   

[tde]

I'm a new SlickEdit customer, and I was also a tad surprised not to find Haskell support out of the box. I would also like to voice my support for Haskell and F#.

And while I'm in request mode, I think distributed version control systems have hit the mainstream. Hardly a day goes by when some impressive project doesn't proudly announce they've migrated their code to DVCS (and shouldn't they have done this years ago). Your sometime competitor, Komodo, included support for Mercurial and Git in their last release and it's very nice. It would be great to have this in SlickEdit too!

[xzenox]

+1 for erlang, you'd get a sale from me as well for it alone. Using emacs is a painful process for at least a number of work days (if not weeks) until you can start having a grasp of things...

I use SE at work on large C++ projects and I love it.

My priority list would be: erlang, scheme

There hasn't been any updates in this thread in a few months, has this "feature request" been formalized yet? Can we expect it in 2010? 2011? Ever?

[ScottW, VP of Dev]

We are aware of the requests for support for these languages. Currently they are not in the release plan--meaning they are not planned for any future release right now. That could change. Generally, we don't announce features like these until we are sure of what we've got.

As you've seen from the comments from the developers, we'd like to do something, but we're hesitant to just offer simple color coding and syntax indenting. We tend to be criticized more heavily for weak support than none at all.

We'll keep talking about this and see what we can do. Sorry that I can't be more definite.

[ydfang]

I would like more SE if it can support ERLang.

otherwise, I need to work with Eclipse also.