What is the Curse of Lisp?

What happens when your language is so powerful that small, independent teams can solve their problems without libraries? Does everyone flock to it? Or do you just get a lack of libraries?

►► Audio, Video, and Transcript available: https://lispcast.com/what-is-the-curs...
►► Subscribe on iTunes: https://itunes.apple.com/us/podcast/t...

Transcript

What is the Curse of Lisp? In this episode, we are going to explore this counterintuitive property of powerful languages. My name is Eric Normand, and I help people thrive with functional programming.

The Curse of Lisp is it has a very personal appeal to me because I used to be one of the people that thought about this all the time. The real question is if Lisp is so powerful, if it's such a great language, and it lets you do so much, why isn't it more popular? It's not about the parentheses. People can get over the syntax. It's been a lot of time to get over the syntax and it's not really what it is. We're going to explore that.

You used to hear this more. In recent years, like the last 10 years, there's been a real renaissance of languages. There's so many possible languages to use now. Mostly because of the Web and the rise of startups who can use whatever language they want on the back end, and there's been a blooming, a blossoming of different possible languages to choose from.

But back in the day it was not like that. Big companies standardized on a mainframe and there were certain languages that ran on that, so they would standardize on a language and train their employees in that language and that's what you had.

There were a lot of -- I'm talking about 50 years ago -- there were a lot of Lisp programmers, mostly at universities and graduate departments, who learned Lisp, got into Lisp, felt very productive in Lisp, and looked around and said, "Look, you've got this hundred-person team working in COBOL and they are producing very little software, you know, just as very slow and here I am with Lisp and I'm like inventing, AI." [laughs]

I'm making stuff happen just all by myself. Maybe I have another person working with me, but like two people doing a lot more work than this team of a hundred people and so it seem to them. This is a perception thing.

The question they would always have is why isn't this more popular? If it's such a powerful language, why aren't businesses investing in it? Why aren't we seeing people realize that they can hire fewer people, a lot fewer people not just 1 or 2 percent, but maybe you slash your team by 80 percent? Why don't we see Lisp being more popular?

Over time, you see this language like C or C++, Java starting to get a lot of attention. A lot of jobs around it. Meanwhile, Lisp is this not growing very much. All this stuff that you could do in Lisp in a few lines of code is now you realize taking men years, person years of work. Sorry. I said men years. I didn't mean to use the sexist version. Person years of work to make a library to do the same thing.

For instance, parsing in a Lisp or any kind of functional language are recursive to send parser, it's very easy to write. Now you need some kind of parser generator system like a Lex and a YACC or an ANTLR in Java. You just need these huge systems, this huge piece of software just to do the same thing that a lisper was able to do.

This is my version of the Curse of Lisp. What happens is because it is hard in Java to do a nice parser, people recognize that it's hard. They work together because the challenge is so great, they need a hundred people to come together and work on it and make it very good and powerful.

It creates even like a cottage industry. Now you need books written about it. You have maybe some conference talks about how to use this new parser generator in Java. It becomes a new thing that you can become an expert at because it's not just like you know how to make parsers. No. You have to know that tool. It's even less general knowledge, but still, it's very powerful because all this work has been put into it.

The question is why is it that the industry seems to prefer this giant effort to make a thing because of a less powerful language and so it takes more effort to do. Why does it prefer that over the more powerful general solution?

I should pause here because it's not about Lisp. I said it at the beginning, it's about powerful languages in general. Any sufficiently powerful, expressive language, like you get with a functional language using higher-order functions and stuff. I think this is going to happen too.