MyClojureQuoraAnswers

Quora Answer : Why is Clojure more popular than Common Lisp when Armed Bear Common Lisp is able to run on the JVM?

Dec 21

Clojure was designed by a very smart developer explicitly to be a good language on the JVM and to interact with the Java ecosystem and all its libraries. And to be good for writing the kind of enterprise data processing applications that he worked on.

Because Rich Hickey wanted a good language for the JVM ecosystem he decided to borrow a bunch of good ideas from Lisp. Because Lisp is a great language.

But he wasn't aiming to make a Common Lisp. This wasn't a language aimed at the Lisp community. And compatibility with Lisp culture wasn't his priority. It was a language aimed at the Java community, to get its work done.

As he was working on a clean slate, and not having to worry about backward compatibility and keeping the Lisp community happy, he could borrow good ideas from elsewhere, too. Ideas that weren't big with the Lisp community.

In particular Hickey picked up two great non-Lisp ideas, and added one very pragmatic idea.

The first great idea Hickey picked up was immutability. Which is a standard virtue in Functional Programming. But while Lisp is the origin of FP. And Lispers probably see referential transparency as a good thing, both Common Lisp and Scheme had generally tried to sell themselves as "multi-paradigm" languages over the years, and therefore had allowed mutability into their culture and code-bases. To make a language with the benefits of modern FP practice, Hickey had to enforce immutability and break with Common Lisp

The second great idea that Hickey picked up was a standard syntax for complex data literals. This doesn't come the FP people, who want to do everything with sophisticated type systems. But comes from the scripting languages like Python, Javascript and Ruby. One of the biggest pains of Java is the difficulty of describing complex nested data literals in your code. Because of the bureaucracy Java tries to enforce, you can only represent and talk about complex nested data-structures if you create a whole bunch of classes and files and data-hiding membranes at every layer of the hierarchy.

This makes sense for some traditional OO programming, but the moment you want to represent something like a chunk of XML it becomes unnecessary hell. Which is why Java programs are accompanied by a swarm of external XML "configuration" files for any literal data they use. Java fanbois try to convince you that this is because data needs to be more flexible than code and so having it in external files that can be edited without recompilation makes sense. This is just a rationalization. Ten percent of the issue is that. The bigger problem (the other 90%) is that Java just can't declare complex data structures, so you either have to laboriously hardwire it together in imperative code or use external files.

Anyone who has moved from Java to Python or Javascript etc. knows the joyful liberation of just being able to talk about data in your program. Because you can assemble any complex nesting of dictionaries / hashes / maps and arrays / vectors etc. as and when you like.

Hickey clearly saw this and decided that he wanted that in his language.

(A secondary benefit is that when you have a good syntax for representing complex data like this, you can ALSO use it for pattern matching and data-extraction. While the scripting languages don't do that, the FP languages do, so Hickey choose the best of both worlds and added some pattern matching capability to his data-representation capabilities)

Now, Lisp people tend to be rather sniffy about anything they consider unnecessary extra syntax. And, understandably, one of the great virtues of Lisp is just how little syntax there is.

However, I believe Hickey had a great insight : while extra syntax for computation (control structures, function invocation etc.) is pretty much always unnecessary. Some extra syntax for the convenience of talking about other kinds of data is a price worth paying. And because he's extra smart, he again managed to synthesize several good ideas. He took JSON which is about as standard a data-structure representation language as you can get. And made a slightly more Lisp-like variation (no commas etc.) He added explicit symbols (ie :keywords). And then it was sufficiently expressive that he could now keep the Lisp virtue of homoiconicity. Instead of EDN (his data notation) being a separate DSL embedded in his Lisp, it was the syntactic infrastructure. Clojure is written in EDN. All valid Clojure syntax is valid EDN and vice versa. And EDN looks enough like JSON that almost any programmer can learn to read it in 5 minutes.

Now Lisp people do understand the value of special syntax for data. But they handle it a different way. They use reader macros to create custom DSLs. This is undoubtedly more powerful than EDN. But having to write a special parser for a custom DSL every time you need a new complex data-structure is almost as bad as Java requiring you to write half a dozen new classes to represent it or write an external XML file. And, as Hickey noted, was likely to lead to a confusion of incompatible DSLs to represent the same data. Having a standard that all Clojure programmers would use is better 99% of the time.

So those are the two great ideas in Clojure : immutability, and a richer underlying syntax that can represent complex data-literals within the program.

Having used them, I would say that any language is better for having made the decision to include these ideas. They are just very good, useful things to have in a language.

Hickey's third idea that breaks with Lisp orthodoxy is what I'd call "pragmatic" rather than "objectively great". Most Lisps have an ideal of being "self-hosting" or bootstrapped / written in themselves. The ideal is to have a very simple core, and create as much of the language in itself as possible.

That is theoretically very satisfying, makes the language very elegant, and can other advantages (eg. work on making, say, the language compiler better pays off twice. It makes the language better, but it also improves the process of making the language)

Lisps are often self-creating / self-sustaining worlds.

However Clojure (and other Clojure-likes) are not like this. Clojure is actually a fairly thin layer on top of Java. The compiler is largely written in Java. Many of the objects in Clojure are actually handled by the equivalents in Java. And Java leaks through the abstraction that is Clojure, more or less by design.

This has downsides and other implications.

For example, Clojure can't do tail-call optimisation, because function calls in Clojure are basically function calls on the JVM. The JVM doesn't do TCO. And Clojure is not implementing its own virtual machine on top of the JVM, so it can't do TCO either. So Clojure has to a new form, the loop, in order to be able to get the TCO-like effect of an iterative loop written kind of like recursion.

Unlike the first two big ideas, this design decision for Clojure (to defer to Java rather than self-host by building its own virtual machine) is not necessarily "great" or something that other languages should copy. But it was, almost certainly "right" for Clojure. It was a pragmatic decision that made Clojure tractable. It makes two-way interop with other JVM code fairly straightforward, and (I guess) efficient. Clojure code is using Java objects, and Java classes and interfaces, and calling Java functions. There's no overhead of an extra layer of Clojure virtual machine.

I'm guessing that Armed Bear, if it is a full Common Lisp, needs to implement more of a VM to get all the Common Lisp goodness.

This decision is undoubtedly a trade-off with both benefits but also costs. In the case of Clojure, the biggest cost I can see is that error reporting it horrendous. And that's because if an error blows your code up at run-time, the machine can't tell which code is really Clojure and which is Java infrastructure. So you get a huge stack-trace mixing up Clojure functions that you recognise with Java functions you didn't even know you were using.

Anyway, this is why (IMHO) Clojure is more popular than Armed Bear or any other implementation of Common Lisp for the JVM.

Clojure is designed by a brilliant language designer who has included two non-Lisp ideas that really shine and make it a great language. Beyond that, it was really designed to "fit in well" with Java and the Java ecosystem. And it does.

No Common Lisp could do what Clojure does. You couldn't introduce immutability without breaking compatibility with Common Lisp. And while I've speculated whether Common Lisp could adopt EDN (Common Lisp for the 21st century seems to have been a project to add similar syntactic convenience) Common Lisp doesn't have it yet.

Finally, Clojure's willingness to get close to Java and the JVM stands in contrast to the self-hosting ideal of Lisp and the values of the Common Lisp community and code-bases.

One day I'd love to see (and work in) a self-hosting Clojure. In fact a full Clojure-machine (Clojure all the way down to the metal). There's nothing preventing that.

But, to be useful and used in the Java world it was probably a good idea for Clojure to make that concession.

Quora Answer : Can Clojure run without the JVM?

Dec 19

Well, Clojure was created as a JVM language.

And some of its design decisions include allowing the lower level JVM to show through in return for having full interoperability with other JVM languages. Eg. when Clojure interacts with Java classes and interfaces that's because Java classes and interfaces are something that's known to it. They are part of Clojure semantics.

ClojureScript is very, very similar to Clojure. Most of the time you won't notice any difference, and the same code is both Clojure and ClojureScript. But it follows a parallel but different philosophy. ClojureScript is designed to interop with Javascript and so it knows and depends on Javascript. But ClojureScript doesn't pretend to know or deal with native Java things.

So, in one sense, ClojureScript is almost like running Clojure without the JVM.

But not quite.

And the authors of both Clojure and ClojureScript are very explicit that they aren't aiming for 100% compatibility between the two. Or for either language to abstract away from, or hide, the underlying host platform.

There was a "Clojure" for the CLR too. I've never used it, and I'm not sure what its current status is. It's almost the same as Clojure except it's designed to work in .NET world. I believe that the philosophy is the same : very like Clojure, but with .NET things as part of its semantics.

There are other languages which are starting to borrow the same philosophy.

I'm interested in Ferret which is Clojure-like but compiles to C++. And is aimed at writing on Arduinos and other small boards.

Again, the ideal is a language with many of the virtues of Clojure, but trading away 100% compatibility in favour of access to the underlying platform. In this case, for example, you can embed short fragments of C directly within Ferret.

Carp is another interesting-looking language for writing native code by compiling to C. It borrows basic Clojure syntax and macros but, as it's aimed as a low-level C replacement, brings in elements of OCAML (a type-system with type inference), and Rust (a borrow-checker).

Hy is more like a "Clojure-flavoured" alternative syntax for Python. But compiles to Python and relies on Python semantics.

In other words, Clojure is rather like Lisp itself, a family of very similar languages with common syntax (in this case like Lisp but built on EDN for extra expressivity about data) but with some subtle variations in vocabulary and semantics.

That means you can have a Clojure-like experience elsewhere. But strictly, Clojure itself is only JVM (or maybe JVM / CLR)

Quora Answer : "Uncle" Bob Martin, author of 'Clean Code' has a strong belief that Clojure could be "The" language of the future. Do you agree?

Aug 22

The way I've heard him say it, Clojure would be a sensible choice as the language we all decided to standardize on the future for most application / enterprise type work.

In this, I totally agree. I like Clojure very much. It is officially "my favouritest language". Or "the best language I know".

I mean I don't know all the languages, but I have been programming for over 30 years and have used quite a few of them. I've programmed professionally in most mainstream languages (and some obscure ones). And in my experience, and not very humble opinion, Clojure is one of the best designed, and nicest to use, all round languages I know. In as far as it's possible to sensibly rank these things, I think it's the "top language"

It is really, really good.

But it's not perfect. And some things are a f****ing PITA. Error reporting is atrocious. Tooling is OK if you can use Emacs but not beginner friendly. Java leaks through. Javascript leaks through ClojureScript.

But it's a great language nevertheless. It has the twin virtues that it's based on Lisp which was already a great language / great language concept. AND it was a "clean slate" reinvention that could make some unorthodox but good breaking changes like EDN syntax and default immutability.

Clojure is superbly well thought, mind-blowingly powerful and a joy to write code in.

Now whether that adds up to "the language of the future" is another question.

Convenience is everything in programming languages. The 3 most important criteria for language success are

1) does it run where I want, and talk to the platform resources I want?

2) does it look exactly like all the languages I know so I don't have to invest too much time learning it?

3) does it claim to fix my bugs?

That's why the language of the future is more likely (and tragically) Typescript. Basically we had C, so people wanted interpreted C that made strings easier and got Perl. Then they wanted Perl in the browser and got Javascript. Now they want Javascript with types ...

OTOH, Clojure

still doesn't have a great Android or mobile story. (It should, in theory, but it turns out it doesn't, and some promising projects from the last few years seem to have fizzled.) It also still doesn't compile well to native for writing native apps. There are interesting Clojure-likes such as nakkaya/ferret and carp-lang/Carp but no official Clojure dialect for native app development)

looks like Lisp and weirds people out. Once you get Lisp it's awesome. But most people want their languages to look like C (or at least something more imperative)

has lots of resources to help fix bugs, but not static types. And type systems are popular. (I confess, I move back and forth between Clojure and Java and I miss being able to define an interface and have the compiler check that I'm sticking to it in Clojure)

Clojure's main problem is that it doesn't have a big tech-giant behind it making it the de-facto language on a platform people want to write for. Interesting Cognitect (ie. Rich Hickey, the inventor and BDFL of Clojure) were just bought by Nubank which might well give Clojure a boost. But unless there's a platform that it's unambiguously the main language for, it's an uphill battle.

Quora Answer : What does Michael O. Church mean when he says "Clojure, on the other hand, certainly makes it possible to write sloppy code; however, it also makes it possible to prevent sloppy code in ways that are very difficult in its static counterparts."?

Mar 19, 2015

Well, I can't speak for Michael O. Church but here's what I would say :

Clojure has no static type-system that will force you to do the right thing. So it's possible to write bad things if you want. But it has the facilities to do the right thing easily, so that you aren't pushed towards badness because of the difficulty of doing things properly.

For example, it's a really good thing to write functions without side-effects. Haskell says that that is SUCH a good idea that it will use the type-system (ie. compile time checking) to make sure you can only write functions with side-effects in a restricted zone of the program.

Clojure doesn't.

Now when I write Clojure, I write very few functions that have side-effects, and I keep them in a well defined place in my code. Most of my libraries are properly side-effect free. So I can do the right thing, no problem. Clojure never fights my desire for referential transparency.

But, of course, sometimes in debugging I'll suddenly want to do an ad-hoc check of an intermediate value of a calculation :

(let [z (f (g x))]

... )

has to become

(let [y (g x)

p (println y)

z (f y)] ... )

I see the value of y, I get enlightened, I fix the bug in g, I put the let block back the way it was. Job done in 3 - 5 minutes (most of which is compiling time). All within the flow.

A more principled language that would stop me doing that, is correct in the longer term : print statements have no place in the let blocks of library functions. But it's wrong for the micro-scale, where you need to be able to bend those rules temporarily to push through your problems.

As to whether Clojure can preventing sloppy code better than Haskell, the most obvious case would probably be something like macros. You can just generalize more of your code out into reusable macros in Lisp than in (ordinary, I haven't tried Template) Haskell. The best code of all is the code you don't write, so having a large chunk of well written and well tested reusability might beat having to write something specific for this situation.